This meta-analysis was conducted in strict accordance with the PRISMA-DTA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Diagnostic Test Accuracy) guidelines [19]. The PRISMA-DTA checklist is provided in Checklist 1. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 for Abstracts checklist is provided in Checklist 2. Search reporting followed the PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension) extension [20], and the completed PRISMA-S checklist is provided in Checklist 3. Additionally, the study protocol has been registered in the PROSPERO (International Prospective Register of Systematic Reviews) database (CRD420251207710). Deviations from the registered protocol comprised an update and expansion of the literature search during revision (final search: August 7, 2026) and the additional calculation of prediction intervals (PIs) for pooled sensitivity and specificity.

We conducted a systematic literature search across 4 databases: PubMed, Embase, Web of Science, and the Cochrane Library, with literature retrieval completed by 2 independent reviewers (ZL and XG) until August 7, 2026. Our search strategy comprised 3 groups of keywords: AI-related terms (eg, artificial intelligence, machine learning, and deep learning); disease-specific terms (eg, esophageal neoplasms, esophageal adenocarcinoma, and ESCC); and neoadjuvant therapy-related terms (eg, neoadjuvant therapy, NACT, neoadjuvant chemoimmunotherapy, and NCRT). A search strategy combining free-text keywords and MeSH was used to ensure accuracy, as detailed in Table S1 in Multimedia Appendix 1. Initial searches posed no restrictions on language or publication year. As an additional information source, citation searching was performed by screening the reference lists of the included studies and of similar published systematic reviews and meta-analyses on the same topic. No study registries or other online resources were searched, no search filters were used, and the search strategy did not undergo formal peer review.

Inclusion criteria carefully adhered to the PITROS (participants, index test, target conditions, reference standard, outcomes, and settings) framework. Participants (P): The participants in this study are patients with esophageal cancer undergoing neoadjuvant therapies, which include NACT, neoadjuvant immunotherapy plus chemoradiotherapy (NICRT), NCRT, and neoadjuvant immunochemotherapy. Index test (I): This study uses AI techniques, specifically ML and deep learning (DL), to analyze imaging data (CT/magnetic resonance imaging [MRI]) to predict the pCR in patients with esophageal cancer. Target condition (T): The positive group is defined as patients with a pathologically confirmed pCR, while the negative group consists of patients who did not achieve a pCR. Reference standard (R): Pathological assessment was used as the reference standard to determine whether patients had achieved pCR. Outcomes (O): The primary outcomes include patient-based analysis of sensitivity, specificity, diagnostic odds ratio (DOR), and the area under the curve (AUC). Setting (S): The study setting includes retrospective or prospective data sources, covering public databases or local hospitals.

Exclusion criteria included studies with titles and abstracts that were clearly unrelated to the research topic, as well as specific types of literature (eg, reviews, case reports, conference abstracts, meta-analyses, and letters to editors). Non-English studies were excluded due to lack of accessibility. This English-only restriction was considered a potential limitation because it may have excluded relevant evidence. Furthermore, research related to tumors at the gastroesophageal junction and studies with overlapping samples were also excluded. The screening process was conducted independently by 2 reviewers (LW and JZ), who first assessed the relevance of titles and abstracts and then evaluated the full texts based on the inclusion and exclusion criteria. Discrepancies among reviewers were resolved through discussion, and if consensus could not be reached, a third reviewer (FR) determined the final decision.

The quality of the included studies was rigorously evaluated using the updated 2025 PROBAST+AI tool (Prediction model Risk of Bias Assessment Tool plus Artificial Intelligence) [21]. This tool provides a comprehensive evaluation of both model development and validation phases, with each phase covering 7 key domains including participants and data sources, predictors, outcome assessment, and analytical methods. Detailed evaluation procedures and results are shown in Tables S2 and S3 in Multimedia Appendix 1.

To ensure objectivity and accuracy, 2 reviewers (ZL and JZ) independently assessed the risk of bias in included studies using the PROBAST+AI tool. Discrepancies were resolved through discussion and consensus, thereby enhancing the reliability and consistency of the final assessments. We used the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) tool to evaluate the certainty of sensitivity, specificity, and DOR, focusing on 5 key domains: risk of bias, indirectness, inconsistency, imprecision, and publication bias [22]. Downgrading in any domain resulted in reduced overall evidence certainty.

Two reviewers (XG and ZL) independently extracted data from all included studies to minimize bias and enhance the accuracy and reliability of the dataset. Any disagreements between the 2 reviewers were resolved through consultation with a third reviewer (LW), who served as an arbitrator to facilitate consensus. Extracted data included the first author, publication year, patient origin country, type of histopathology, study design, type of therapy, reference standard, No. of pCR patients, and analysis approach. Technical information was also recorded, including type of imaging, imaging technical details, data source, type of AI model, data splitting method, AI method, the best-performing AI algorithms, as well as the number of patients in the training, internal validation, and external validation sets, with corresponding counts of true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs). For studies lacking data required for meta-analysis, we contacted corresponding authors via email to request the missing information.

The primary outcomes included sensitivity, specificity, AUC, and DOR for both internal and external validation sets. Internal validation referred to evaluation within the development data or a held-out subset, whereas external validation referred to evaluation in an independent cohort from a different institution or time period and therefore provided a more direct assessment of generalizability. Sensitivity (TP rate) was used to assess the ability of an AI model to correctly identify pCR cases and was calculated as TP/(TP+ FN). Specificity (TN rate) reflected the probability of correctly identifying non-pCR cases and was calculated as TN/(TN+ FP). The AUC served as a comprehensive measure of the model’s ability to distinguish between positive and negative cases. DOR, calculated as the ratio of the positive likelihood ratio (LR+) to the negative likelihood ratio (LR−), was used to evaluate diagnostic performance. For studies with multiple nonoverlapping patient datasets (eg, internal validation and external validation), each dataset was treated as an independent dataset and included in the analysis. Additionally, for studies evaluating multiple AI models, only the model with the highest AUC from the internal or external validation sets was extracted. Internal and external validation were synthesized separately. Validation sample counts were summed within each group; training samples and repeated cross-validation folds were not added again. The 2 validation groups contributed 2668 validation observations in total (1891+777). This sum was not interpreted as a verified count of unique patients because patient overlap across validation datasets and publications could not be fully assessed from the available study-level reports.

A bivariate random-effects model was applied in the meta-analysis to assess the diagnostic performance of AI models for predicting pCR in patients with esophageal cancer. Because sensitivity and specificity were jointly modeled, the bivariate random-effects framework was retained as the primary model rather than splitting the outcomes into separate univariate analyses solely to apply the Hartung-Knapp-Sidik-Jonkman (HKSJ) correction; HKSJ is primarily designed for conventional univariate random-effects meta-analysis and is not a general replacement correction for a bivariate diagnostic-accuracy model. AUC was derived from the summary receiver operating characteristic (SROC) curve, and DOR was derived from the difference between the model coefficients for logit sensitivity and logit FP rate; conventional HKSJ was therefore not directly applicable to these derived outcomes. Predictive results were pooled for the internal and external validation cohorts. Forest plots were generated to visually present pooled sensitivity, specificity, and DOR, while an SROC curve displayed the summary point and joint confidence and prediction regions. Z-tests were used to compare pooled sensitivity, specificity, DOR, and AUC values, with statistical significance defined as P<.05. Inconsistency was assessed using outcome-specific I² values based on Cochran Q statistic. Q was calculated for logit sensitivity, logit specificity, and log DOR. For sensitivity and specificity, 0.5 was added to both event and nonevent counts if either count was zero. For DOR, 0.5 was added to all 4 cells if any cell was zero. I² estimated the proportion of observed variability attributable to between-study differences rather than sampling error, not the absolute magnitude of heterogeneity. Between-study variance (τ²), its square root (τ), and PIs were obtained from the bivariate model to describe variation on the corresponding effect scales [23]. Meta-regression analyses considered AI method (ML vs DL), AI model (radiomic and clinical models vs radiomic-only models), and contrast-enhanced imaging (yes vs no), using the categories reported in the supplementary meta-regression tables. Bubble plots were generated to assess temporal trends in AI model AUCs, and radar charts were constructed to depict the distribution of ML and DL algorithms. The clinical utility of the AI model was evaluated using a Fagan nomogram, with a pretest probability of 34%, close to the median pCR rate across 43 internal and external validation datasets (34.8%; IQR 27.7%-40.6%). Although PIs were not specified in our original PROSPERO registration protocol, we additionally calculated marginal 95% PIs for sensitivity, specificity, and DOR in both validation groups to better quantify the distribution of effects across different populations and settings, as recommended by recent methodological guidance on the reporting of PIs [24,25]. Potential small-study effects and funnel-plot asymmetry were assessed using Deeks test; asymmetry was not interpreted as evidence of publication bias alone [26,27]. The primary bivariate estimates, PIs, and model-selection sensitivity analysis were obtained using R (R Foundation for Statistical Computing; version 4.5.2). AUC CIs for the primary analyses were estimated using the delta method. Secondary analyses used Stata (StataCorp LLC; versions 15.1 and 17.0) with the Midas and Metadta modules. Data visualization used the ggplot2 and tidyverse packages in R.

Sensitivity analyses excluded outliers identified in bivariate box plots, excluded low-quality studies, and included only ESCC cohorts (Table 1). We also compared the highest-AUC model with the first eligible imaging model reported in each article, using the same validation cohorts and pCR definitions (Tables S9 and S10 in Multimedia Appendix 1). Models were selected in table order, or in Results order when no eligible performance table was available; comparisons were limited to datasets with usable counts for both selections.

The search strategy yielded 1680 records, including 411 from PubMed, 734 from Embase, 521 from Web of Science, and 14 from the Cochrane Library. After removing 693 duplicates using EndNote 20 (Clarivate), 987 records were screened by title and abstract, and 919 were excluded because they were case reports, letters, reviews, animal experiments, meta-analyses, or clearly irrelevant. Following a detailed full-text review of 68 articles, 35 studies were further excluded for nonimaging-based AI model research (n=8), inaccurate definition of pCR (n=12), gastroesophageal junction cancer with unclear pathology (n=7), unavailable diagnostic data (TP, FP, TN, FN; n=4), or overlapping samples (n=4). In addition, citation searching of the reference lists of similar published meta-analyses (ie, the studies included in those reviews) identified 113 additional records; after screening, 104 were removed as irrelevant, and the remaining 9 full-text articles were excluded because of sample overlap. Ultimately, 33 articles were included in this systematic review [16,28-59]. The literature screening process was systematically documented using a standardized PRISMA flow diagram, as shown in Figure 1. We identified 17 non-English or mixed-language reports; none met the other eligibility criteria.

A total of 33 eligible studies [16,28-59] were included. Internal validation was reported in 27 studies comprising 27 datasets and 1891 patients [16,28-34,36-38,40,41,43,44,46,48,49,51-59], while external validation was reported in 16 studies with 16 datasets involving 777 patients [16,33-37,39,41-43,45,47,50,53,55,57]. Nearly all studies were conducted in China (32/33) [16,28-53,55-59], with one study [54] from Japan, and studies were published between 2019 and 2026. Most included studies were retrospective, using pathological results as the reference standard. CT was the most commonly used imaging modality (30/33, 91%), most commonly with contrast enhancement (28/33, 85%). In both internal and external validation sets, ML was the predominant approach for building AI models. Logistic regression (LR) was the most frequently used algorithm in internal validation sets (9/27, 33%), whereas Extreme Gradient Boosting (XGBoost) was the most frequent in external validation sets (4/16, 25%). A detailed summary of the study, patient, and technical characteristics is presented in Table 2 and Tables S4-S6 in Multimedia Appendix 1. Ten publications contributed to both validation groups, yielding 43 validation datasets across 33 publications. The 2 validation counts summed to 2668 observations (1891+777); the number of unique patients across datasets and publications could not be verified.

The PROBAST+AI quality evaluation tool was applied to assess the risk of bias, with results presented in Figure 2 and Tables S2 and S3 in Multimedia Appendix 1. For model development, 3.0% (1/33) [51] of studies were rated as high risk in the overall judgment of quality, and 3.0% (1/33) [54] showed high concern regarding applicability. For model validation, 36.4% (12/33) [30,31,35,39,42-44,46,48,51,52,58] of studies were assessed as high risk in the overall risk of bias judgment, and 3.0% (1/33) [54] demonstrated high concern regarding applicability. Overall, high-risk items were relatively infrequent, with low-risk items predominating, indicating that the quality of the included literature was generally acceptable.

Certainty of evidence, assessed using GRADE criteria, was very low for all GRADE-assessed outcomes (Table 3). The main reasons for downgrading were a high risk of bias in multiple studies, unexplained inconsistency, indirectness, and imprecision of the CIs.

In the internal validation sets, LR was the most frequently used AI algorithm (9/27, 33%), whereas XGBoost was the most frequent in the external validation sets (4/16, 25%; Figures 3A and 3B). Across internal validation sets, AUC values showed a slight increase from 2019 to 2026 (Figure 3C). By contrast, AUC values in external validation sets showed a downward trend over the same period, indicating that the generalizability of AI models remains a key concern (Figure 3D).

Comparisons of pooled diagnostic performance are shown in Figure 4. For internal validation, the pooled sensitivity of AI models was 0.77 (95% CI 0.72- 0.81; very low certainty), and the pooled specificity was 0.82 (95% CI 0.78-0.86; very low certainty; Figure S1A in Multimedia Appendix 1 and Table 3). The 95% PIs were 0.57-0.89 for sensitivity and 0.60-0.93 for specificity (Figure S1A in Multimedia Appendix 1). The pooled DOR was 15.49 (95% CI 11.16-21.50; very low certainty; Figure 5A and Table 3), and the pooled AUC was 0.86 (95% CI 0.83-0.89; Figure S2A in Multimedia Appendix 1). The DOR 95% PI was 5.40-44.44; the pooled estimate described average performance rather than a universal clinical value. Descriptive I² values were 37.88%, 66.10%, and 34.19% for sensitivity, specificity, and DOR, respectively.

In the external validation sets, the pooled sensitivity was 0.73 (95% CI 0.61-0.81; very low certainty), and the pooled specificity was 0.83 (95% CI 0.73-0.90; very low certainty; Figure S1B in Multimedia Appendix 1 and Table 3). The 95% PIs were 0.34-0.93 for sensitivity and 0.36-0.98 for specificity (Figure S1B in Multimedia Appendix 1). The pooled DOR was 12.93 (95% CI 7.25-23.07; very low certainty; Figure 5B and Table 3), and the pooled AUC was 0.84 (95% CI 0.79-0.89; Figure S2B in Multimedia Appendix 1). The DOR 95% PI was 2.40-69.62. Descriptive I² values were 51.78%, 78.38%, and 42.28% for sensitivity, specificity, and DOR, respectively.

Statistical comparison revealed no significant differences between internal and external validation sets in sensitivity (P=.47), specificity (P=.83), DOR (P=.60), or AUC (P=.50; Figure 4A). Statistical significance should not be equated with clinically meaningful improvement; clinical relevance also depends on heterogeneity, evidence certainty, calibration, decision thresholds, and clinical consequences.

Algorithm-specific estimates are shown in Table 4 and Figure S3 in Multimedia Appendix 1. Results from a single dataset are explicitly identified as single-study estimates, with CIs reflecting within-study sampling uncertainty; they are not pooled estimates. The small number of datasets in several subgroups precluded reliable ranking of algorithms. For different imaging modalities, CT-based models achieved a pooled sensitivity of 0.75 (95% CI 0.70‐0.80), specificity of 0.82 (95% CI 0.78‐0.86), and AUC of 0.86 (95% CI 0.82‐0.88). MRI-based models showed a pooled sensitivity of 0.79 (95% CI 0.59‐0.91), specificity of 0.86 (95% CI 0.67‐0.95), and AUC of 0.91 (95% CI 0.79‐1.00). No statistically significant differences were observed between the 2 modalities for sensitivity (P=.64), specificity (P=.59), or AUC (P=.37; Figure 4A). NACT had the numerically highest pooled AUC (0.88, 95% CI 0.77‐0.98) among the treatment subgroups (Figure 4B).

Meta-regression analyses indicated that, in the internal validation sets, performance was associated with AI method (ML, 21 datasets, vs DL, 6 datasets; sensitivity P=.01, specificity P<.001) and AI model (radiomic and clinical models, 8 datasets, vs radiomic-only models, 19 datasets; P<.001 for both), whereas the use of contrast-enhanced imaging was not statistically significant (P=.13 and P=.11; Table S7 in Multimedia Appendix 1). Bivariate box plots further identified the studies by Wang et al [28], Qi et al [29], Qi et al [30], Fan et al [31], Liu et al [32], and Qi et al [33] as potential contributors to the observed heterogeneity (Figure 6A).

In the external validation sets, the meta-regression did not identify statistically significant sources of heterogeneity (ML, 12 datasets, vs DL, 4 datasets: P=.13 and P=.97; radiomic and clinical models, 3 datasets, vs radiomic-only models, 13 datasets: P=.81 and P=.37; contrast-enhanced imaging: P=.83 and P>.99; Table S8 in Multimedia Appendix 1). Bivariate box plots suggested that Zhang et al [34], Ling et al [35], and Zhao et al [36] may represent potential sources of heterogeneity (Figure 6B).

Table 1 and Figure S4-S6 in Multimedia Appendix 1 summarize the results of the sensitivity analyses. These analyses were performed separately by excluding low-quality studies, removing outliers identified in bivariate box plots, and including only studies on ESCC. In the internal validation sets, sensitivity ranged from 0.76 to 0.77, specificity from 0.82 to 0.83, AUC from 0.85 to 0.86, and DOR from 15.15 to 16.06 across the sensitivity analyses, indicating that the results were stable. In the external validation sets, sensitivity ranged from 0.71 to 0.74, specificity from 0.81 to 0.83, AUC from 0.84 to 0.85, and DOR from 10.72 to 13.76, suggesting that the results were consistent.

Thirty of 33 [16,28-40,42-48,50-53,55-59] publications reported multiple eligible imaging models. Comparing the highest-AUC model with the first eligible imaging model reported in each article was possible for 16 internal and 10 external datasets; the selected model changed in 4 and 3 datasets, respectively. Average performance decreased modestly with the alternative selection (Tables S9 and S10 in Multimedia Appendix 1).

Deeks funnel plot test showed no evidence of funnel-plot asymmetry suggesting small-study effects in either the internal validation sets (P=.70) or the external validation sets (P=.50; Figures 6C and 6D). Based on a prior probability of 34%, the Fagan nomogram for the internal validation sets showed a positive post probability of 69% and a negative post probability of 13% (Figure 7A). For the external validation sets, the Fagan nomogram indicated a positive post probability of 69% and a negative post probability of 15% (Figure 7B).