Our study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement [14]. The protocol has been registered in the International Prospective Register of Systematic Reviews (CRD42024583829). The PRISMA checklist is shown in Multimedia Appendix 1.

The following studies were included: (1) those that developed complete ML models constructed by radiomics for the diagnosis and prognostic prediction of ICC; (2) original studies that conducted only internal validation without external validation; we also acknowledge the contributions of articles without external validation and have summarized the results of both the training and validation sets, which can help check for potential overfitting (therefore, these studies were also included); (3) English articles; and (4) case-control, cohort, or cross-sectional research.

The following reports were excluded: (1) meta-analyses, reviews, guidelines, expert opinions, or conference abstracts released with no peer review; (2) those that performed image segmentation without complete radiomics models for the diagnosis and prognostic prediction of ICC; and (3) those lacking one of outcome measures as follows for evaluating model accuracy: C-index, SEN, specificity (SPC), accuracy, precision, confusion matrix, diagnostic 4-fold table, or F₁-score.

PubMed, Web of Science, Embase, and the Cochrane Library were retrieved from their inception until August 23, 2024. MeSH (Medical Subject Headings) and free-text terms were used, with no restraint on region or publication year. The strategy is shown in Table S1 in Multimedia Appendix 2.

Reviewers independently uploaded the searched records into EndNote and excluded duplicate entries. Subsequently, they reviewed all titles and abstracts, selected relevant original studies, downloaded full texts, and further screened them to identify the eligible studies.

A standard data extraction form was created for extracting specific information: the first author, publication year, country, design, patient source, outcome events, radiomics source, segmentation methods, imaging study participants, region of interest segmentation software, total case number, the number of cases in the validation or training group, validation set–generating method, and model type. In addition, outcome measures were extracted to assess model accuracy, including C-index, SEN, SPC, accuracy, precision, confusion matrix, and diagnostic 4-fold table. Literature screening and information extraction were separately implemented by 2 investigators (LX and ZC) and then cross-check was implemented. If any dissent arose, a third analyzer (YW) would address them.

The study quality is assessed via the Radiomics Quality Score and the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). The Radiomics Quality Score includes 16 items, with each study section corresponding to a specific project. Scores were assigned based on the methodological quality and summed to generate an overall score for each project. The range of total scores was from –8 to +36. Scores between –8 and 0 were considered 0%, and a score of 36 denoted 100% [15].

QUADAS-2 has been extensively applied in systematic reviews of diagnostic accuracy studies across various medical fields. The tool evaluates 4 domains in terms of risk of bias and concerns regarding applicability, with 3 possible categorizations: high risk, unclear risk, and low risk. The assessment of risk of bias specifically includes 4 key items: patient selection, the index test under evaluation, the reference standard’s implementation and interpretation, and patient flow. Each item comprises 3 specific assessment questions. If any of these questions are rated as high risk, the corresponding item is classified as high risk [16]. Two investigators (LX and ZC) appraised the quality of the eligible studies. Upon completion, they cross-checked the results. If any disagreements arose, a third researcher (YW) was consulted for the final decision.

In this systematic review, the outcome measures were the area under the receiver operating characteristic curve (AUROC), SEN, and SPC, which reflected the accuracy of ML models constructed by radiomics.

A meta-analysis was conducted on the C-index, an indicator of the overall accuracy of ML models. When 95% CI and SE of the C-index were lacking, the SE was conjectured based on the method put forward by Debray et al [16]. The heterogeneity index (I²) was used to assess interstudy heterogeneity. When I² surpassed 50%, a random-effects model was used for analysis. Conversely, when I² was less than 50%, a fixed-effects model was applied.

Moreover, a bivariate mixed-effects model was leveraged to implement the meta-analysis of SEN and SPC according to diagnostic 4-fold tables. Nevertheless, most original studies offered no such tables. In such cases, the diagnostic 4-fold tables were derived based on SEN, SPC, precision, and case numbers. The meta-analysis was implemented leveraging Stata (version 15.0; StataCorp).

Initially, 2218 studies were identified, of which 329 were duplicates. Next, after 1009 records were removed for other reasons, we evaluated the titles and abstracts and found 69 potentially eligible reports. Subsequently, 3 studies with no outcomes of interest and 8 conference abstracts were removed. Eventually, 58 studies were included. Figure 1 shows the PRISMA flowchart.

Fifty-eight studies were included, comprising 15 cohort studies and 41 case-control studies, with 2 sourced from public databases. Of all studies, 40 were single-center studies, 8 were dual-center studies, and 10 were multicenter studies. Sample sizes ranged from 36 to 764 cases. These studies focused primarily on the diagnosis, risk stratification, and prognostic prediction of ICC.

Thirty eligible studies reported the diagnostic performance of radiomics-based ML for ICC; 6 studies evaluated overall survival (OS) in patients with ICC; 8 studies assessed MVI; 9 studies investigated recurrence; 5 studies analyzed gene mutations; 2 studies reported perineural invasion (PNI); 2 studies evaluated LN positivity; 2 studies assessed tertiary lymphoid structures (TLSs); 2 studies reported progression-free survival (PFS); 1 study examined noncurative resection; 3 studies assessed recurrence-free survival; 3 studies evaluated tumor grading; 1 study addressed radiographic response in patients with ICC undergoing transarterial radioembolization; and 1 study focused on vascular endothelial growth factor and cytokeratin 7.

Across all studies, 13 studies used deep learning (DL), while 45 used ML approaches. Among these, images were manually segmented in 44 studies, semiautomated segmentation was adopted in 5 studies, and automatic segmentation was leveraged in the remaining 9 DL-based studies. ITKSNAP (Penn Image Computing and Science Laboratory, University of Pennsylvania) and LifeX (IMIV or CEA, Université Paris-Saclay) were the most commonly used software tools for region of interest segmentation. A total of 47 studies carried out internal validation, while 12 studies implemented external validation. Most studies (n=29) assessed radiomics performance using computed tomography (CT), while approximately one-third (n=19) relied on magnetic resonance imaging (MRI). Moreover, 1 study delved into CT combined with MRI, while 2 studies leveraged positron emission tomography (PET) or CT. Finally, 7 studies used ultrasonography (US). The specific types of radiomics varied significantly across studies. In diagnostic models, logistic regression was mostly adopted, whereas Cox regression was primarily used in prognostic prediction models. Other algorithms, such as random forests, support vector machines, neural networks, and XGBoost, were also used. Since the number of incorporated studies was large, these algorithms were not elaborated in detail (Table S2 in Multimedia Appendix 2).

Among the 58 eligible studies, 9 studies had no explicit description of imaging protocols, resulting in a score of 0 for the Image Protocol Quality criterion. In 2 studies, segmentation was not conducted by different physicians, algorithms, or software, nor did they perform segmentation with (random) noise interference or across varying respiratory cycles, yielding no points for the multiple segmentation item. Twenty-two studies did not acquire images at different time points, leading to a score of 0 for this aspect. Forty-four studies incorporated multivariable analyses with nonradiomic features, such as Ki-67 mutation, which could potentially provide more comprehensive models; however, these studies did not evaluate or discuss biological relevance. Nineteen studies performed cutoff value analyses, reducing the risk of reporting overly optimistic results, earning 1 point for this criterion. Among all included studies, 40 reported discriminatory statistics and their statistical significance, while 18 additionally used resampling techniques, earning 1 and 2 points, respectively. For calibration statistics, 32 studies reported calibration and its statistical significance, and 2 applied resampling techniques. All eligible studies were not prospectively registered in a trial database, resulting in a score of 0 for this criterion. Four studies lacked validation, scoring –5 points. Thirty-eight studies conducted validation using datasets from the same institution, earning 2 points. Two studies performed validation using datasets from another institution, scoring 3 points. Nine studies performed validation using datasets from 2 different institutions, scoring 4 points, while 5 studies performed validation with datasets from 3 or more institutions, earning 5 points. Only 3 studies assessed the concordance of their models with current “gold standard” methods, earning points in this domain. Seventeen studies reported the current or possible clinical applications, whereas only 2 included cost-effectiveness analyses in a clinical setting. For open-access segmentation of the regions of interest, 53 studies scored 1 point, while 5 studies did not make any code or data publicly available. As a result, the scores of the included studies varied between –2 and 14, with a mean score of 9.21 (25.6%). Further details are shown in Table S3 in Multimedia Appendix 2. Details of the QUADAS-2 assessment are shown in Figures 2 and 3. Our assessment revealed that 17 studies exhibited a high risk of bias in the patient selection domain, while 4 studies demonstrated a high risk of bias regarding applicability concerns, as evaluated via the QUADAS-2 tool.

Our study uncovers that radiomics is extensively used in ICC, for instance, in the diagnosis of ICC, MVI, LN positivity, genetic mutations, PNI, TLSs, and prediction of recurrence. Among these, the primary focus is on diagnosing ICC, identifying genetic mutations, detecting MVI, and predicting recurrence, wherein radiomics exhibited relatively favorable accuracy. In the validation group, the C-index, SEN, and SPC of models constructed by CFs for detecting ICC were 0.762 (95% CI 0.728-0.796), 0.72 (95% CI 0.66-0.77), and 0.72 (95% CI 0.66-0.78). The C-index, SEN, and SPC of radiomic-based models were 0.853 (95% CI 0.824-0.882), 0.80 (95% CI 0.73-0.85), and 0.88 (95% CI 0.83-0.92). The C-index, SEN, and SPC of models based on both radiomics and CFs were 0.912 (95% CI 0.889-0.935), 0.77 (95% CI 0.72-0.81), and 0.90 (95% CI 0.86-0.92).

Previous studies have explored various diagnostic tools for ICC. A 2019 meta-analysis assessed the robustness of cytokeratin-19 fragment (CYFRA21-1) for diagnosing ICC and reported an AUROC of 0.904 (SE 0.0171), SEN of 0.81 (95% CI 0.75-0.86), and SPC of 0.86 (95% CI 0.82-0.89) [21]. A retrospective study investigating the prognostic effects of CT and MRI features on ICC included 204 patients who underwent curative ICC resection. It identified enhancement patterns and infiltrative tumor margins as predictors of OS and event-free survival, while biliary invasion was determined to be a predictor of OS [22]. You and Yun [23] identified 11 MRI characteristics for distinguishing hepatocellular carcinoma (HCC) from ICC. Among these, valuable MRI characteristics with high diagnostic odds ratios (DORs >20) included arterial rim enhancement, progressive enhancement, and a target appearance on hepatobiliary phase images. In addition, a meta-analysis [24] compared the performance of MRI and contrast-enhanced ultrasonography (CEUS) to differentiate ICC from HCC. For MRI, the SEN, SPC, DOR, and AUROC were reported as 0.81 (95% CI 0.79-0.84), 0.90 (95% CI 0.88-0.91), 41.47 (95% CI 24.07-71.44), and 0.93 (95% CI 0.90-0.96). For CEUS, the SEN, SPC, DOR, and AUROC were 0.88 (95% CI 0.84-0.90), 0.80 (95% CI 0.78-0.83), 42.06 (95% CI 12.38-133.23), and 0.93 (95% CI 0.87-0.99). Although CEUS demonstrated higher pooled SEN, MRI showed superior pooled SPC. However, comprehensive reviews addressing recurrence and MVI are scarce, with most research focusing on these aspects in HCC [25-27]. A meta-analysis on CT radiomics in anticipating MVI in HCC reported pooled SEN, SPC, and AUROC values of 0.82 (95% CI 0.77-0.86), 0.79 (95% CI 0.75-0.83), and 0.87 (95% CI 0.84-0.91) [25]. Radiomics-based ML demonstrated favorable performance in detecting MVI in ICC in our analysis. In the validation cohort, the C-index, SEN, and SPC of models developed by both radiomics and CFs were 0.836 (95% CI 0.792-0.881), 0.77 (95% CI 0.67-0.84), and 0.88 (95% CI 0.79-0.94). The models developed solely on radiomics or CFs should be further validated and used in clinical practice.

In existing studies, the primary imaging modalities include CT, PET-CT, MRI, and US. The NCCN Clinical Practice Guidelines [28] recommend the use of MRI and CT in combination with clinical evaluation for diagnosis. In actual clinical practice, MRI and CT are the most commonly used imaging techniques. Our study revealed that in the validation cohort, models constructed by MRI and CFs outperformed those based on CT or US combined with CFs in diagnosing ICC, with C-index values of 0.922 (95% CI 0.896-0.949), 0.896 (95% CI 0.853-0.939), and 0.902 (95% CI 0.835-0.969), respectively. MRI exhibited a highly superior diagnosis performance in this context. Furthermore, subgroup analyses based on alternative outcomes in our study yielded variable results. MRI, CT, and other modalities showed optimal efficacy depending on the specific outcome, as detailed in Tables S4-S19 in Multimedia Appendix 2. These findings suggest that the selection of imaging modalities can be tailored to specific diagnostic objectives without additional, unnecessary imaging modalities.

Image segmentation has emerged as a critical process in radiomics. Segmentation techniques can be broadly categorized into manual segmentation and DL-based automatic segmentation [29]. Among the studies included in our analysis, 45 used traditional ML approaches, of which only 2 used semiautomatic segmentation, while the remainder relied exclusively on manual segmentation. In DL-related studies, 1 used manual segmentation. Overall, current radiomics workflows are predominantly dependent on manual segmentation, which poses several challenges. First, the vector knowledge and reproducibility of the individuals performing manual segmentation require further evaluation. The segmentation variability arising from different operators performing manual segmentation possibly introduced uncertainty in the evaluation of radiomics models, thereby affecting the reproducibility of the model. In the future, the application of artificial intelligence for automatic segmentation or the adoption of more standardized segmentation protocols will mitigate the impact of manual segmentation. Second, the dependency on variations among different manufacturers of the same equipment and different imaging parameters for the same image should be assessed. However, the studies included in our analysis did not conduct preliminary experiments in this regard. Third, the impact of different imaging phases should be further discussed. Among the eligible studies, only a limited number of studies described images obtained at different phases, such as arterial, venous, delayed, or mixed phases. Since studies reporting such outcomes were insufficient, further analysis on this was not undertaken.

The models based on radiomics primarily consist of traditional ML and DL models [30]. In traditional ML, texture features need to be extracted from segmented regions, and features should be selected for model construction. However, the feature selection is challenging, primarily due to the complexity of diverse feature selection methodologies and the potential loss of raw image information. These factors may compromise the accuracy of models based on ML [31]. Moreover, in the case of a large number of features, it is complicated to eliminate the risk of overfitting during the modeling process [32]. In our study, the C-index between the 2 sets was relatively similar, which indicated that the risk of overfitting was effectively mitigated in the current research. In the context of DL, the processes of image feature extraction and texture selection are integrated into the learning process, thereby preserving the integrity of image information to a greater extent [33]. Our findings indicate that DL demonstrates superior performance in comparison with traditional ML. In the diagnosis of ICC, radiomics-based DL models markedly outperformed other models in the validation cohort. The C-index values of CT, MRI, and US were 0.937 (95% CI 0.893-0.982), 0.968 (95% CI 0.903-1.033), and 0.891 (95% CI 0.786-0.996), respectively. Furthermore, among models constructed by both radiomics and CFs, subgroup analysis based on US-derived images uncovered that the C-index of DL models was 0.924 (95% CI 0.863-0.984), notably higher than other approaches. However, these results are derived from a limited set of models, and further exploration in future studies is warranted to substantiate these findings.

In ML research, the validation methods of models have drawn significant attention. Most existing studies primarily adopted internal validation to assess models. In internal validation, the training data and validation data were derived from the same dataset, with various random sampling methods, such as random sampling, k-fold cross-validation, leave-one-out, and bootstrap, used to generate the validation set. This process introduced certain biases, as the dataset and validation set distributions were relatively similar, which might have limited the generalizability of the model. In addition, biases possibly arose during the random sampling process. External validation refers to situations where the dataset and validation set came from different datasets, particularly with differences in factors such as underlying conditions. External validation demonstrated the model’s generalizability and verifiability. Particularly in radiomics, external validation was worth attempting due to challenges stemming from image parameters and vector knowledge from region of heterogeneity segmentation, which could differ significantly between datasets. If a model performed exceptionally well in external validation, it indicated high robustness. In this review, it was found that most radiomics studies in ICC primarily focused on internal validation, lacking substantial external validation to support their feasibility. Consequently, this imposed certain limitations in interpreting the results. Although 12 studies based on external validation were included, it was found that radiomics in ICC involved many task types, and few external validation studies existed for each task type. Therefore, subgroup analysis of the validation set generation method was not performed, nor was there further discussion.

Moreover, the studies included manufacturers such as Siemens, Philips, and others, but some original studies did not specify the exact manufacturers in detail. In addition, some studies included images from multiple manufacturers. Due to limitations in the number of studies and varying reporting methods, an in-depth discussion of radiomics applications in ICC under different or multiple manufacturers was not conducted. Parameters may have affected the results, but the categories and quantities of parameters included in the studies varied widely, and many studies did not present imaging protocols or acquisition parameters. As a result, the discussion on imaging parameters is also limited and is shown only in the tables.

Our study shows that radiomics demonstrates ideal results in the diagnosis and prognosis of ICC and holds significant promise for future clinical practice. However, we find that integrating radiomics into PACS systems to enable an intelligent diagnostic process still presents challenges. First, the segmentation of radiomics images is a crucial process, and most current studies rely on manual segmentation. As a result, differences in segmentation exist depending on the experience and expertise of the radiologists, which may impact the model’s fitting or application performance. Therefore, prioritizing DL-based automatic segmentation is imperative. To improve interpretability, the development of more highly automated ML models or DL diagnostic systems represents a promising direction for future research. In conclusion, future research should consider the accuracy of image segmentation after integration into PACS systems, develop intelligent tools to achieve highly accurate image segmentation, and promote the intelligent reading of subsequent imaging data. Furthermore, potential regulatory issues will need to be considered, including how to avoid patient information leakage and address ethical concerns related to new systems and processes introduced during evaluation.

There are several limitations in our study. First, gray literature was not included due to its specific circulation channels, which may lead to publication bias. Moreover, the heterogeneity in all studies could not be fully excluded. In addition to CT types and features, as well as ML models, the heterogeneity may be attributed to variations in image segmentation, feature extraction, pathological types, and modeling algorithms. Nevertheless, because of limited studies in these subgroups, it was not possible to elucidate these detailed features. Therefore, the quantitative analysis results should be interpreted carefully. In radiomics research, the public disclosure of feature selection, dimensionality reduction techniques, and hyperparameter tuning enhanced the transparency and readability of the studies. Since many original studies did not provide detailed information on hyperparameter tuning, including manufacturer and parameter descriptions, this research listed only the specific details in tables without further discussion. This represented another limitation of existing radiomics studies [34,35]. Third, although plenty of papers were included, subgroup analyses were conducted based on different models, image sources, and tasks. Hence, a limited number of original studies were included for certain types of tasks and images. Due to relatively few studies on some models, it is difficult to discuss the performance differences among various models. Finally, few studies compare the diagnostic performance of radiomics with that of radiologists, and thereby, it is infeasible to elucidate the accuracy of radiomics versus human assessments.

Our study unravels that radiomics is broadly used in ICC, particularly in diagnosing ICC, MVI, genetic mutations, PNI, LN positivity, and TLSs, and predicting recurrence and OS. It is a noninvasive technique for tumor diagnosis. However, due to the reliance on manual segmentation, equipment parameters, and expert knowledge, further studies should aim to include more cases. Moreover, research on DL in this field is still insufficient. Future studies should incorporate more cases and focus on identifying heterogeneity. It is also essential to develop intelligent image-based reading programs for ICC, which could act as an adjunct technique for the diagnosis and treatment of ICC.