We comprehensively searched multiple databases (PubMed, Embase, and Web of Science) with an initial cutoff on March 25, 2025, and a supplementary search in May 2025 to ensure inclusion of the latest research. The search strategy incorporated 3 keyword groups: the first group comprised AI-related terms (such as “artificial intelligence,” “radiomic,” and “deep learning”), the second group focused on target-related terms (including “early recurrence” and “tumor recurrence”), and the third group included disease-related terms (such as “cholangiocarcinoma”). We used a combined approach of free-text terms and Medical Subject Headings, with no initial restrictions on language or publication year. Detailed search strategy information is provided in Multimedia Appendix 2. Additionally, we manually reviewed the reference lists of included studies to identify further relevant literature.

Studies were included based on the Participants, Index test, Target condition, Reference standard, Outcomes, and Setting (PITROS) framework. For comprehensive details, please consult Textbox 1.

Additionally, we systematically excluded studies with obviously irrelevant titles and abstracts, as well as inappropriate literature types, including reviews, conference abstracts, case reports, meta-analyses, and letters to editors. Furthermore, non-English studies were excluded due to inaccessibility. Moreover, articles that did not explicitly specify CT-based methodologies and those from which true positive (TP), false positive (FP), true negative (TN), and false negative (FN) values could not be obtained were also excluded.

We used an improved version of the revised Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) assessment protocol to systematically evaluate the methodological characteristics of the studies [21]. QUADAS-2 is specifically designed to assess bias risk and applicability in diagnostic studies, with its earlier versions having been widely applied across various research contexts. However, we identified certain irrelevant criteria that were not applicable in our context and incorporated the Prediction Model Risk of Bias Assessment Tool (PROBAST) to improve and adapt the risk of bias assessment standards for predictive models [22]. The modified QUADAS-2 tool encompassed 4 primary domains: participants, index test (ie, the applied AI algorithm), reference standard, and analysis. Detailed scoring criteria are presented in Multimedia Appendix 3. Moreover, for certainty rating, the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) tool was used to assess the evidence level for each standard [23], with scoring details provided in Multimedia Appendix 4. To ensure the objectivity and accuracy of the assessment process, 2 independent reviewers (JC and JX) comprehensively evaluated the bias risk of included studies. During the review process, any disagreements between reviewers were resolved through in-depth discussion and analysis.

The 2 reviewers (JC and JX) independently conducted preliminary screenings of the remaining articles’ titles and abstracts to determine their potential eligibility. In cases of disagreement, a third reviewer (KL) would serve as a supervisor to assist in reaching a consensus. The extracted data included patient and study-level information (first author’s name, publication year, patient origin country, type of cholangiocarcinoma, reference standard, lesion-based or patient-based analysis, training set, and total number of patients in internal and external validation sets). Imaging technology–level information included CT type, AI method, AI algorithm, data segmentation method, and TP, TN, FP, and FN values in training, internal, and external validation sets. TP referred to the outcome where the AI model judged as positive based on CT and confirmed as early recurrence of cholangiocarcinoma through pathology or clinical imaging follow-up. TN referred to the AI model judging as negative based on CT and the reference standard confirming the absence of cholangiocarcinoma (ie, no disease present). FP indicated the AI model judged as positive based on CT but confirmed as negative by reference standard (eg, misdiagnosed as a disease but actually a benign lesion or normal variation). FN represented cases where the AI model judged as negative based on CT but confirmed as positive by reference standard (ie, missed disease cases). For studies included in the systematic review but lacking data available for meta-analysis, we sent emails to corresponding authors to obtain the necessary information.

Given the limited availability of diagnostic contingency tables in the majority of studies, we implemented 2 primary methodological approaches to construct 4-cell diagnostic matrices. First, we retroactively calculated TP, FP, TN, and FN using sensitivity, specificity, and the number of positive cases under the reference standard and total patient numbers. Second, through receiver operating characteristic curve analysis, we used GetData Graph Digitizer software (S Fedorov) to replot points, extracting the optimal sensitivity and specificity based on the best Youden index, and then retroactively calculated TP, FP, TN, and FN in combination with the number of positive cases under the reference standard and total patient numbers. In studies providing multiple nonoverlapping patient datasets, we assumed these contingency tables to be independent and therefore extracted them in full. For studies presenting AI models with multiple algorithms, we also performed a comprehensive extraction to enhance the feasibility of comparing algorithms between different approaches.

The primary outcome measures included sensitivity, specificity, diagnostic odds ratio (DOR), and AUC from both internal and external validation sets. Sensitivity (also known as recall or TP rate) measures the ability of the AI model to accurately identify true cases, calculated by the formula TP/(TP+FN). Specificity (also known as TN rate) reflects the model’s capability to correctly identify healthy cases, calculated by the formula TN/(TN+FP). AUC, a comprehensive indicator for evaluating the model’s ability to distinguish between positive and negative cases. DOR is a comprehensive diagnostic performance metric that combines sensitivity and specificity. This metric quantifies the diagnostic test’s discriminative capability by comparing the probability of a positive result in populations with disease versus without disease.

Using a bivariate random-effects statistical framework, we comprehensively analyzed the diagnostic capabilities of AI applied to CT imaging for early cholangiocarcinoma recurrence detection. We separately summarized diagnostic results for internal and external validation sets, calculating pooled sensitivity, specificity, and DOR values. Forest plots showed combined sensitivity and specificity, and a summary receiver operating characteristic curve illustrated the 95% confidence and prediction ranges, with pooled AUC values calculated.

To assess interstudy heterogeneity, we used the Higgins I² statistic, where I² values of 25%, 50%, and 75% represent low, moderate, and high heterogeneity, respectively. For substantial heterogeneity (I²>50%), we conducted subgroup analysis and meta-regression to explore potential sources of heterogeneity. Included subgroup analysis and meta-regression variables comprised type of cholangiocarcinoma, analysis type, reference standard, type of CT, AI method, AI model, and data splitting method. Additionally, we used bubble plots to assess the variation of AI model DOR values over time. We also used Fagan plots to evaluate the clinical application impact of AI models. Furthermore, we used Deeks’ funnel plot asymmetry test, assessing publication bias through effective sample size and DOR logarithmic regression. Statistical analyses were performed with Midas in Stata (version 15.1; StataCorp LLC), and risk of bias was assessed using RevMan (version 5.4; Cochrane). All tests were 2-sided, with P<.05 considered significant.

A total of 3 initial database searches discovered 368 potentially relevant literatures. After removing 105 duplicate articles, 263 articles entered the preliminary screening phase. During the initial screening, 238 articles were excluded due to irrelevant titles and abstracts and inappropriate literature types, leaving 25 articles for full-text review. After detailed examination, we excluded 1 study with insufficient or incomplete diagnostic data (including TPs, FPs, FNs, and TNs), 9 studies not based on CT-based AI models, and 6 studies that failed to evaluate cholangiocarcinoma recurrence in patients. Ultimately, 9 studies met the inclusion criteria and were incorporated into the meta-analysis [17,24-31]. The literature screening process followed the standardized PRISMA protocol, which is detailed in Figure 1.

A total of 9 eligible studies were identified, with internal validation sets comprising 8 studies and 13 datasets [17,25-31], encompassing 1234 patients. External validation involved 5 studies with 17 datasets [24,26,27,29,30], totaling 303 patients. These included studies were published between 2021 and 2023. A total of 78% (7/9) of the studies were conducted in Asia (Japan=1 and China=6), with the remaining 22% (2/9) in North America (United States=2). All 9 studies were retrospective. Overall, 5 studies used pathology and clinical imaging follow-up as the reference standard [24,26,27,29,31], while 4 studies used clinical imaging follow-up alone as the reference standard [17,25,28,30]. Additionally, 8 studies focused on IHC [17,24-28,30,31], while 1 study addressed EHC [29]. Furthermore, 8 studies conducted patient-based analysis [24-31], and 1 study performed image-based analysis [17]. Regarding data splitting methods, 4 studies used random splitting [28-31], 4 studies used cross-validation [17,25-27], and 1 study divided data based on 2 independent hospitals [24]. In terms of AI approaches, 8 studies used machine learning [24-31] and 1 study used deep learning [17]. For internal validation sets, the most frequently used modeling methods were random forest (2/13, 15%) and logistic regression (2/13, 15%). For external validation sets, the most common modeling methods included light gradient boosting machine (LightGBM; 3/17, 18%), logistic regression (2/17, 12%), random forest (2/17, 12%), neural network (2/17, 12%), Bayesian classifier (2/17, 12%), support vector machine (2/17, 12%), and extreme gradient boosting (XGBoost; 2/17, 12%). Study, patient, and technical characteristics are summarized in Table 1 and Multimedia Appendix 5.

Bias risk was assessed using the revised QUADAS-2 tool, as shown in Figure 2. In patient selection, 2 studies were rated as “high risk.” Bo et al [24] inappropriately excluded patients with Child-Pugh scores >7, and Chen et al [26] excluded patients with performance status scores >2 or Child-Pugh scores >7. Overall, high-risk items were few, with low-risk items predominantly represented, indicating that the included studies were acceptable in overall quality (Multimedia Appendix 3). According to GRADE standards, the evidence quality assessment for outcome indicators ranged from very low to low, suggesting weak certainty of evidence (Multimedia Appendix 4).

Emerging computational intelligence technologies, particularly machine learning and deep learning, have demonstrated remarkable progress through enhanced algorithmic designs and increased data accessibility. In internal validation sets, the DOR remained relatively stable from 2021 to 2023, with some algorithms (residual network 50 [ResNet50] and XGBoost) achieving higher DOR values, while others maintained lower levels (Figure 3A). In external validation sets, DOR values showed a slow increase during the same period, with some algorithms (random forest and LightGBM) achieving higher DOR values, suggesting potential improvements or refinements in AI algorithms (Figure 3B).

In internal validation sets, ResNet50 simultaneously achieved the highest sensitivity (0.98) and specificity (0.94). In external validation sets, the neural network demonstrated the highest sensitivity (0.94), while the support vector machine showed the highest specificity (0.88), as detailed in Table 2.

In the internal validation sets, the CT-based AI model demonstrated a sensitivity of 0.87 (95% CI 0.81-0.92; very low certainty), specificity of 0.85 (95% CI 0.79-0.89; very low certainty), and a DOR of 37.71 (95% CI 18.35-77.51; very low certainty), as shown in Figures 4 and 5A. Additionally, the AUC of the model was 0.93 (95% CI 0.90-0.94; Figure 6A). Based on the predefined 20% test probability, the Fagan nomogram revealed a positive likelihood ratio of 59% and a negative likelihood ratio of 4% (Multimedia Appendix 6).

Meta-regression revealed high heterogeneity in sensitivity (I²=99.43%) and specificity (I²=99.58%) for internal validation sets. The meta-regression analysis results showed no identifiable sources of potential heterogeneity (Table 2). Subgroup analysis demonstrated no statistically significant differences in sensitivity and specificity of CT-based AI models across various categories, including type of cholangiocarcinoma, analysis, reference standard, AI model, AI method, and data splitting method (all P>.05; Table 3).

In the validation sets conducted externally, the AI model based on CT exhibited a sensitivity of 0.87 (95% CI 0.81-0.91; low certainty) and a specificity of 0.82 (95% CI 0.77-0.86; very low certainty; Multimedia Appendix 7). Additionally, it showed a DOR of 30.81 (95% CI 18.79-50.52; very low certainty; Figure 5B) and an AUC measuring 0.85 (95% CI 0.82-0.88; Figure 6B). When applying a 20% pretest probability, the Fagan nomogram indicated a positive likelihood ratio of 55% along with a negative likelihood ratio of 4% (Multimedia Appendix 6).

No statistically significant variances were found between the internal and external validation sets regarding sensitivity (z score=0.00; P>.99), specificity (z score=0.87; P=.38), and DOR (z score=0.00; P=.08). However, the AUC for the internal validation set was significantly greater than that of the external validation set (z score=4.35; P<.001).

Meta-regression revealed high heterogeneity in sensitivity (I²=42.07%), and although specificity (I²=0%) did not show high heterogeneity, we attempted to identify potential sources of heterogeneity. Meta-regression analysis results indicated that different reference standards (specificity, P<.001) and type of cholangiocarcinoma (sensitivity, P=.05) might be potential sources of heterogeneity. In the cholangiocarcinoma-type subgroups, the sensitivity was 0.88 (95% CI 0.83-0.92) for IHC and 0.74 (95% CI 0.50-0.98) for EHC, with IHC showing significantly higher sensitivity compared to EHC (P=.05). In the recurrence reference standard subgroups, the specificity of pathology combined with clinical imaging follow-up was 0.83 (95% CI 0.78-0.87), which was significantly higher than that of clinical imaging follow-up alone (0.78, 95% CI 0.68-0.88; P<.001; Multimedia Appendix 8).

For the internal validation sets, after excluding low-quality studies, the AI model’s sensitivity was 0.90 (95% CI 0.80-0.95), specificity was 0.84 (95% CI 0.71-0.91), DOR was 43.43 (95% CI 12.31-153.31), and AUC was 0.93 (95% CI 0.91-0.95). For the external validation sets, after excluding low-quality studies, the AI model’s sensitivity was 0.88 (95% CI 0.70-0.96), specificity was 0.76 (95% CI 0.44-0.84), DOR was 21.79 (95% CI 5.74-82.68), and AUC was 0.80 (95% CI 0.77-0.84; Multimedia Appendix 9).

For the internal validation sets, the bivariate box plot suggested that Wakiya et al [17], Song et al [30], and Zhu et al [31] may represent potential contributors to statistical heterogeneity. After excluding the studies identified in the bivariate box plot, the AI model’s sensitivity was 0.83 (95% CI 0.78-0.87), specificity was 0.82 (95% CI 0.75-0.88), DOR was 22.28 (95% CI 14.43-34.40), and AUC was 0.87 (95% CI 0.84-0.90). For the external validation sets, the bivariate box plot suggested that Chen et al [26] and Hao et al [27] might be potential sources of heterogeneity (Multimedia Appendix 10). After excluding the studies identified in the bivariate box plot, the AI model’s sensitivity was 0.88 (95% CI 0.82-0.93), specificity was 0.80 (95% CI 0.74-0.85), DOR was 30.48 (95% CI 17.27-53.79), and AUC was 0.87 (95% CI 0.84-0.90).

Using Deeks’ funnel plot methodology, we identified substantial publication bias in the internal validation cohorts (P<.001), whereas for the external validation sets, there was no evidence of small-study effects (P=.25; Figures 7A-7B).

Our study demonstrated the exceptional performance of CT-based AI models in diagnosing early recurrence of cholangiocarcinoma. In the internal validation sets, the model exhibited high diagnostic performance (sensitivity=0.87, specificity=0.85, DOR=37.71, and AUC=0.93). However, a moderate decline in performance was observed during external validation (sensitivity=0.87, specificity=0.82, DOR=30.81, and AUC=0.85), with a statistically significant reduction in AUC (P<.001). This finding suggests that despite the model’s excellent performance in internal datasets, its generalizability across different patient populations and imaging conditions may be limited. The outstanding performance of the CT-based AI model primarily stems from its powerful feature extraction capabilities, enabling it to capture high-dimensional radiomics features that reflect tumor heterogeneity and microenvironmental characteristics [32]. These features are particularly crucial for highly invasive and heterogeneous cholangiocarcinoma, which are often challenging to detect through conventional imaging techniques. By using advanced algorithms, such as machine learning and radiomics, the model can effectively identify complex patterns within imaging data, thereby enabling precise prediction of early recurrence risk in cholangiocarcinoma [29,33,34]. However, the performance decline observed in external validation highlighted several potential limitations. First, heterogeneity in CT acquisition techniques across different institutions (such as scanner models, contrast agent protocols, and reconstruction parameters) may compromise the model’s stability [35]. Second, variability in patient demographics and pathological characteristics may further contribute to decreased performance in external cohorts. These findings underscore the importance of conducting large-scale, multicenter external validation studies to ensure the robustness and generalizability of AI models across different clinical environments [36].

Our subgroup analysis results based on different cholangiocarcinoma subtypes revealed that in the external validation sets, the sensitivity of IHC was significantly higher than that of EHC (P=.05). A similar trend was observed in the internal validation sets, although the difference was not statistically significant. This difference may be attributed to the distinct biological characteristics and imaging features of IHC and EHC. IHC typically demonstrates pronounced tumor heterogeneity and microenvironmental characteristics, which are more readily captured by AI algorithms on CT imaging, thereby enhancing diagnostic sensitivity. In contrast, the anatomical complexity of EHC and its less distinctive imaging characteristics may limit the diagnostic performance of AI models [37,38]. It is important to note that the EHC dataset was limited (only 1 group of data), which may constrain the model’s learning capacity for EHC. Future multicenter studies are needed to validate these findings.

In 2023, Yang et al [39] conducted a meta-analysis evaluating the diagnostic performance of machine learning models for early recurrence of IHC. The analysis included 5 studies comprising a total of 1247 patients, using paired and network meta-analysis to compare the diagnostic accuracy of machine learning models with traditional clinical models. The results demonstrated that machine learning models exhibited a pooled diagnostic sensitivity of 0.92, a specificity of 0.79, and an AUC of 0.81, revealing superior diagnostic value compared to traditional clinical models. In contrast, our AI model demonstrated higher diagnostic performance for predicting early recurrence of cholangiocarcinoma in the internal validation set, with an AUC value of 0.93. This may be attributed to the larger sample size included in our study, as well as the use of more advanced deep learning algorithms. However, it is important to note that, in contrast to the previous meta-analysis, our study focused solely on CT-based AI models and used a more specific data source for model training.

In 2025, Xu et al [40] conducted a meta-analysis evaluating the application of machine learning based on radiomics in IHC. Their results showed that AI diagnostic models integrating radiomics and clinical features achieved a sensitivity of 0.85 and specificity of 0.77. Our study showed that the CT-based AI model achieved a sensitivity of 0.87 and a specificity of 0.85. Compared to previous research, our diagnostic performance was superior, which may be related to differences in patient populations and AI models included. Xu et al [40] focused primarily on IHC cases, whereas our study included both IHC and EHC. Additionally, our analysis exclusively evaluated CT-based AI models. Compared with previous meta-analyses, our unique strength lies in merging different specific algorithms separately and stratifying the study population into internal and external validation sets to assess the generalizability of the AI models, thus providing more rigorous and comprehensive diagnostic evidence.

The high heterogeneity of included studies might affect the overall sensitivity and specificity of AI models in internal and external validation sets. For internal validation sets, we sought to identify sources of heterogeneity through meta-regression and box plot analysis. While meta-regression did not identify any significant factors, box plots indicated that studies by Wakiya et al [17], Zhu et al [31], and Song et al [30] might be primary sources of heterogeneity. In the external validation sets, meta-regression indicated that cholangiocarcinoma types and reference standards were potential sources of heterogeneity, while box plots also identified studies by Chen et al [26] and Hao et al [27] as significant contributors to heterogeneity. Nevertheless, high heterogeneity may still result from multiple interacting factors, including patient age and status, tumor stage, geographical region, preprocessing methods, classification algorithms and validation approaches, sample size, and AI model characteristics, such as feature selection, hyperparameter optimization, and modeling algorithms [41-44]. The complex interplay of these factors may cause differences between studies, highlighting the need for future research to standardize variables and reduce heterogeneity for more accurate and reliable results. To address the challenges posed by heterogeneity, future studies should consider implementing harmonization techniques, which standardize imaging parameters and processes used across different institutions. Robust data augmentation methods can enhance the diversity of training datasets, enabling models to generalize effectively across diverse populations. Additionally, domain adaptation strategies can facilitate the transfer of learned features from source datasets to new, unseen populations, thereby enhancing model robustness and generalizability [45].

Our results demonstrate that CT-based AI models achieve high diagnostic performance in internal validation sets and moderate performance in external validation sets. AI can perform preliminary reading, enabling clinicians to process cases more quickly, improve turnaround time, expand the accessibility of specialized reports, and ultimately alleviate pressure on the health care system. Implementing CT-based AI models in primary health care systems, such as general practice, may lead to early disease detection and timely intervention. However, it is worth emphasizing that these models should not be viewed as independent standards or decision-making tools, but rather as useful resources in emergency situations (when expert consultation is unavailable) or for residents and clinicians lacking expertise in detecting this disease. In addition, none of the studies included in our research reported the comparative performance of AI-assisted human interpretation against that of AI alone, making it challenging to assess the specific added value of AI tools for human readers. Additionally, the studies did not detail how AI models perform relative to radiologists, which may limit the practical interpretability of the results. Future research could benefit from exploring comparative analyses that examine the interplay between AI and radiologists. Moreover, while our study primarily focused on AI models based on CT imaging, it is important to acknowledge the inherent limitations of relying solely on this modality. CT imaging does not account for the comprehensive insights that can be gained from MRI and ultrasound imaging techniques. Each of these imaging modalities has unique strengths; for example, MRI excels in soft tissue contrast and provides functional information, while ultrasound offers real-time imaging capabilities and is more accessible in certain clinical settings. Future AI model evaluations should consider integrating cross-modal imaging data, associating observations with patients’ clinical backgrounds, and effectively communicating synthesized insights through reports [46,47].

Additionally, our study primarily focused on traditional machine learning algorithms, with 8 of the 9 included studies using methods like logistic regression, random forests, and support vector machines. Only 1 study used a deep learning approach (ResNet50). Deep learning models typically use image data as their primary input, allowing them to automatically learn hierarchical features from raw data. In contrast, traditional machine learning models rely on parametric values and features that are often manually crafted from the data [48]. Although deep learning excels at handling complex, high-dimensional data, its “black box” nature may result in a lack of interpretability and transparency in clinical settings, which could affect its practical acceptability and the reproducibility of model decisions [49]. Therefore, future research should incorporate a greater number of deep learning models to comprehensively assess their effectiveness and applicability relative to traditional algorithms. At the same time, the importance of explainability in the clinical deployment of AI models cannot be overstated. Future studies should incorporate interpretable frameworks, such as Gradient-Weighted Class Activation Mapping and Shapley Additive Explanations, and attention maps, to ensure that AI-generated predictions can be understood by clinicians [50]. This not only fosters trust in AI systems among health care providers but also facilitates their broader adoption in real-world clinical environments [51].

Our study has several limitations. First, all included studies used retrospective designs, which may introduce potential bias. Well-designed prospective studies are needed to further validate our meta-analysis findings. Second, not all studies defined the gold standard as histopathological examination, which could influence diagnostic performance. However, our subgroup analysis showed no statistically significant differences in sensitivity or specificity among different gold standards, suggesting that the choice of gold standard may have a limited impact. Third, most of the included studies (7/9, 78%) were conducted in Asian regions, which may limit the generalizability of our conclusions.

Our meta-analysis systematically evaluated the diagnostic performance of CT-based AI models for predicting early recurrence of cholangiocarcinoma. The results indicate that while these models demonstrate high diagnostic accuracy in internal validation cohorts, their performance is only moderate in external validation cohorts. Significant heterogeneity between studies may affect the robustness of these findings. Future research should prioritize prospective study designs and the adoption of standardized reference standards to validate these findings and improve the clinical utility and generalizability of AI models.