A systematic literature search was conducted in PubMed, Embase, and Web of Science without date restrictions to identify relevant studies published up to November 9, 2024. The search strategy included 4 groups of terms: those related to AI (eg, “artificial intelligence,” “machine learning,” “deep learning”), cervical cancer (eg, “uterine cervical neoplasms,” “cervix,” “cervical”), lymph-vascular space invasion (eg, “LVSI,” “lymphatic vessel invasion,” “lymphatic permeation”) and imaging modalities (eg, “MRI,” “magnetic resonance imaging,” “PET/CT,” “positron emission tomography-computed tomography”). Both keywords and Medical Subject Headings terms were used, and complete strategy could be seen in Multimedia Appendix 1. We manually reviewed the reference lists of selected studies for additional articles and repeated the search on December 11, 2024, to ensure the inclusion of recent publications.

The selection of studies adhered to the PICOS (Population, Intervention, Comparison, Outcome, Study Design) framework to ensure rigorous assessment and relevance.

Studies were excluded if they met any of the following conditions: (1) irrelevant titles or abstracts; (2) focused solely on lymph node metastasis rather than LVSI; (3) inappropriate article types, including reviews, conference abstracts, case reports, or meta-analyses; case reports were excluded due to their limited sample size, typically involving only a single patient, which makes them unsuitable for statistical analysis or model evaluation; (4) studies with incomplete or uninterpretable data, where TP, FP, FN, and TN values for internal and external validation sets could not be extracted; and (5) animal or in vitro studies. The screening process was conducted in duplicate by 2 independent reviewers (LS and YL), who first evaluated titles and abstracts for relevance. Full-text articles were then assessed against the inclusion and exclusion criteria. Duplicate articles were identified and removed using EndNote’s (Clarivate) duplicate detection tool, followed by manual verification. Discrepancies between reviewers during the screening process were resolved through discussion, and if consensus could not be reached, a third reviewer (HW) was consulted to make the final decision.

To comprehensively assess the quality of included studies, we adapted a recognized tool, the revised Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) [24], by replacing certain criteria with more relevant ones from the Prediction Model Risk of Bias Assessment Tool (PROBAST) [25]. The rationale for this adaptation lies in the complementary strengths of the 2 tools. QUADAS-2 is widely recognized for evaluating the risk of bias in diagnostic accuracy studies, but it does not fully address methodological considerations specific to prediction models, such as variable selection, outcome definitions, or overfitting. PROBAST, on the other hand, is specifically designed to assess risk of bias in prediction model studies. By combining elements from both tools, we tailored the modified QUADAS-2 tool to account for the unique characteristics of studies exploring image-based AI models for LVSI detection.

Our modified QUADAS-2 tool evaluates 4 domains: participants, index test (AI algorithm), reference standard, and analysis. In addition to assessing the risk of bias across each domain, we also evaluated applicability concerns in the first 3 domains. Two independent reviewers applied the modified QUADAS-2 tool to assess the risk of bias in each study, resolving any disagreements through discussion.

Two independent reviewers assessed study eligibility and performed data extraction, resolving any discrepancies through consensus, with a third reviewer serving as an adjudicator if needed. Data extraction encompassed the primary author’s name, publication year, study design, and country of origin. Additionally, key elements such as imaging modality, reference standard, and total and LVSI+ counts of patients, lesions, or images in training, internal validation, and external validation sets were recorded. Further details included age, AI method, AI model algorithms, and performance outcomes, specifically the TP, FP, TN, and FN counts for both internal and external validation sets. Internal validation refers to the evaluation of model performance using a subset of data from the same source as the training data. External validation, in contrast, involves using an independent dataset that originates from a different source than the training set.

The principal outcome measures used in this analysis were sensitivity, specificity, and AUC for both internal and external validation sets. Sensitivity was defined as the proportion of TP scans among the total number of positive cases (TP+FN) for patients, images, or lesions. Specificity was defined as the proportion of true negative (TN) scans among the total number of negative cases (TN+FP). The AUC, representing the area under the receiver operating characteristic curve, served as a summary measure of the model’s diagnostic ability. We extracted AI performance data from internal validation sets and external validation sets. For studies reporting multiple machine learning or deep learning algorithms, only the best diagnostic performance model or algorithm (with the highest AUC value) was included as a representative of the study in the meta-analysis.

A bivariate random-effects analytical approach was used to generate pooled estimations of sensitivity and specificity outcomes for AI-based assessments of LVSI in both internal and external validation sets, accompanied by 95% CIs. We used a summary receiver operating characteristic model to generate the summary receiver operating characteristic curve and calculate the AUC. Additionally, Fagan plots were created to provide a visual representation of the clinical utility of the models.

To assess heterogeneity among studies, we calculated the I² statistic, interpreting values of 25%, 50%, and 75% as indicating low, moderate, and high heterogeneity, respectively. For internal validation sets, we performed subgroup analyses by the number of patients (>150 vs ≤150), region (single vs multiple centers), AI method (deep learning vs machine learning), AI algorithm (logistic regression vs support vector machine), and imaging modality (MRI vs PET/CT). Publication bias was evaluated using Deeks’ funnel plot. All statistical analyses were conducted in Stata 15.1, with significance set at P<.05. Risk of bias for study quality was assessed using RevMan 5.4 (The Cochrane Collaboration) for comprehensive risk of bias assessment.

A systematic literature retrieval and analysis was methodically executed across 3 authoritative databases: PubMed, Embase, and Web of Science, yielding 568 studies. After removing 165 duplicates, 383 studies were excluded based on initial screening criteria. Subsequently, 20 full-text articles were further assessed. In total, 4 studies were excluded due to unavailable data (TP, TN, FP, and FN, n=1), non-English language (n=1), or not focusing on LVSI (n=2). Ultimately, 16 studies met all inclusion criteria and were included in the final analysis [17-20,26-37]. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart detailing this selection process is shown in Figure 1.

This meta-analysis included 16 studies, all of which included an internal validation set with a total of 1092 participants (ranging from 25 to 198 participants per study). In total, 4 studies included an external validation set [29,30,32,37], involving 203 participants (ranging from 26 to 102 participants). In total, 15 studies used MRI as the imaging modality, while 1 study used PET/CT [37]. Machine learning techniques were applied in 15 studies, with 1 study using deep learning [27]. Among the AI model algorithms, LR was the most common AI model algorithm, used in 9 studies [17,19,28,29,32-35,37], followed by support vector machine (SVM) in 5 studies [18,20,30,31,36], and decision tree and convolutional neural network algorithms, each used in 1 study [26,27]. The gold standard for diagnosis was all pathological examination. All studies were retrospective and conducted in China. Study and patient characteristics, as well as technical features, are summarized in Tables 1-3.

Quality assessment was performed using the QUADAS-2 revised tool, as shown in Figure 2 [9,17,18,20,26-37]. For patient selection, one study was classified as “high risk” due to inappropriate exclusion criteria that may omit specific populations, such as those with a history of radiotherapy or chemotherapy [29]. One study [19] was rated as “unclear” because it was uncertain whether the patients were included consecutively. Regarding the index test in risk of bias, one study was rated as “high risk” for inadequate reporting of model training and evaluation processes [32]. Overall, the quality of the included studies was deemed acceptable.

For internal validation sets, the sensitivity in detecting LVSI of cervical cancer was 0.84 (95% CI 0.79‐0.87), and the specificity was 0.79 (95% CI 0.75‐0.82; Figure 3 [9,17,18,20,26-37]), with an AUC of 0.88 (95% CI 0.84‐0.90; Figure 4A). Using a pretest probability of 20%, the Fagan nomogram demonstrates a positive likelihood ratio of 49% and a negative likelihood ratio of 5% (Figure 5A).

For external validation sets, the sensitivity in detecting LVSI was 0.79 (95% CI 0.70‐0.86), and the specificity was 0.76 (95% CI 0.67‐0.83; Figure 6 [29,30,32,37]), with an AUC of 0.84 (95% CI 0.81‐0.87; Figure 4B). Using a pretest probability of 20%, the Fagan nomogram demonstrates a positive likelihood ratio of 45% and a negative likelihood ratio of 6% (Figure 5B).

In the subgroup analysis, we evaluated diagnostic performance across the number of internal validation patients, region, AI methods, AI algorithms, imaging modalities, and data source (Table 4). Number of internal validation patients (>150 vs ≤150) showed no statistically significant differences in sensitivity (P=.35) and specificity (P=.08). Single-center studies demonstrated significantly different specificities compared to multicenter studies (P<.001). Deep learning (1 study) exhibited higher sensitivity compared to machine learning (15 studies, P=.01), with no significant differences in specificity (P=.29). Among AI algorithms, LR (9 studies) and SVM (5 studies) showed comparable diagnostic performance, with no significant differences in sensitivity (P=.77) or specificity (P=.36). Imaging modality analysis revealed a significant difference in sensitivity between MRI (15 studies) and PET/CT (1 study, P=.01), while specificity remained consistent (P=.29). The radiomic model showed a higher sensitivity, and the radiomic and clinical model showed higher specificity; the differences were statistically significant (P=.05 for both sensitivity and specificity comparisons).

Deeks’ funnel plot asymmetry test showed that there is no significant publication bias both for the internal validation sets (P=.07; Figure 7A) and external validation sets (P=.09; Figure 7B) for AI.

Based on our comprehensive literature review, this is the first meta-analysis comprehensively evaluating imaging-based AI for detecting LVSI in cervical cancer. The pooled sensitivity, specificity, and AUC in our analysis for interval validation were 0.84, 0.79, and 0.88, respectively, while for external validation, these metrics were slightly lower at 0.79, 0.76, and 0.84. The slightly lower performance in external validation compared to internal validation is expected when applying AI models to independent datasets. This decline may be due to differences in patient populations, imaging protocols, data quality, and clinical practices across institutions. Despite this, the proposed AI models still show reasonable generalizability. They demonstrate potential for robust performance in diverse datasets and clinical settings.

Deep learning showed superior sensitivity (0.88) compared to machine learning (0.79), likely due to its advanced capabilities in processing complex imaging data. Deep learning models, particularly convolutional neural networks, excel at autonomously extracting hierarchical features from imaging datasets, revealing subtle patterns that manual feature extraction often misses [27]. And by integrating diverse imaging variables like tissue texture, morphological characteristics, and intensity variations, deep learning enables more precise LVSI-positive case differentiation [27,38,39]. However, given the limited number of deep learning studies, the observed higher sensitivity may be insufficiently generalizable. Larger-scale, multicenter studies are essential to validate the potential superiority of deep learning approaches in LVSI detection. Additionally, deep learning is challenged by high overfitting risk and significant computational demands, which can hinder its practical application in clinical settings [27,40].

Comparative imaging analysis of LVSI detection revealed differential sensitivity across modalities. MRI-based AI demonstrated a pooled sensitivity of 0.79, whereas PET/CT-based AI reported a marginally superior sensitivity of 0.88. The potential enhanced diagnostic performance of PET/CT can be attributed to its sophisticated capacity for integrating functional and anatomical data, thereby facilitating more nuanced metabolic and structural characterization [41]. However, it is important to note that only 1 study used PET/CT in this analysis, and the limited sample size may have influenced the reported sensitivity. This warrants cautious interpretation of the results and highlights the need for further studies with larger PET/CT-based datasets to confirm these findings.

In 2024, Zhang et al [22] conducted a meta-analysis specifically examining MRI-based radiomics models for predicting LVSI in cervical cancer. Their findings reported a sensitivity of 0.79, specificity of 0.73, and an AUC of 0.83. In contrast, our study achieved superior results in the internal validation set, with a sensitivity of 0.84, specificity of 0.79, and an AUC of 0.88. The improved performance in our research could be attributed to the incorporation of more recent studies and a larger sample size, which likely enhanced the accuracy and robustness of the model. Similarly, a recent meta-analysis by Zhao et al [21] reported comparable diagnostic performance metrics, with a sensitivity of 0.83, specificity of 0.74, and an AUC of 0.86. Unlike their studies focusing only on MRI, our meta-analysis integrated multiple imaging modalities, including MRI and PET/CT, thus offering a more comprehensive evaluation of AI-based diagnostic approaches. Moreover, we pioneered the approach of stratifying the analysis into internal and external validation cohorts, enabling a more rigorous evaluation of the models’ diagnostic performance and generalizability. The slight decline in external validation cohorts indicates acceptable generalizability and provides clinicians with realistic expectations regarding model performance in real-world clinical scenarios.

Our meta-analysis demonstrated no significant heterogeneity in both internal and external validation datasets. However, the Deeks’ funnel plot revealed a borderline P value of approximately 0.07, suggesting the potential presence of publication bias. Therefore, we used a bivariate random-effects model for sensitivity and specificity pooling, acknowledging inherent clinical heterogeneity and potential bias. Subgroup analyses were strategically conducted based on the number of patients, region, AI method, AI algorithms, imaging modalities, and data source. The results indicated that both the region and the AI methods significantly influenced the pooled outcomes. However, several additional factors may also contribute to clinical heterogeneity, including variations in AI algorithm architectures, hyperparameter optimization strategies, image acquisition protocols, preprocessing techniques, feature selection methods, and discrepancies in LVSI assessment criteria among pathologists. As reported in Huang et al [17], variations in image acquisition protocols, such as differences in imaging resolution or contrast enhancement methods, may impact the diagnostic performance. Similarly, heterogeneity arising from AI algorithm design, such as the use of different convolutional neural network algorithms, has been shown to influence overall results [27].

Despite these challenges, our findings revealed promising diagnostic performance of AI-based approaches for LVSI detection in cervical cancer, suggesting their potential to streamline clinical workflows and enhance diagnostic accuracy. The implementation of AI-driven approaches in cervical cancer surveillance has increased over the years, yielding encouraging therapeutic prospects [42]. The implementation of imaging-based AI tools could particularly benefit primary health care systems, especially in resource-limited settings where specialist expertise may be scarce. However, challenges include limited computing infrastructure, data storage constraints, and the need for tailored training programs [43]. Additionally, AI systems could optimize screening efficiency and support treatment planning decisions [44]. There is the lack of direct comparisons between the performance of AI models and that of radiologists. Such comparisons are critical to understanding the relative strengths and weaknesses of AI in clinical practice. Future studies should aim to investigate head-to-head comparisons between AI algorithms and radiologists.

There are several limitations in our meta-analysis. First, external validation was performed in only 4 of 16 studies, raising concerns about model generalizability. The limited external validation increases the risk of overfitting, where models may perform exceptionally well on training datasets but demonstrate reduced performance on independent datasets. Second, the inherently retrospective design of the included studies may introduce selection biases and limitations in data collection. Second, all studies were conducted in China, which raises concerns about the generalizability of our findings to other populations and health care settings. These factors underscore the critical need for future prospective investigations that not only address the biases associated with retrospective designs but also validate and strengthen the robustness of our findings across diverse demographics and clinical environments. Third, the absence of direct comparative analyses between AI and radiologist interpretations represents a significant research gap; further comparisons are needed.

In conclusion, imaging-based AI, particularly deep learning algorithms, demonstrates promising diagnostic performance in predicting LVSI in cervical cancer. However, the limited external validation datasets and the retrospective nature of the research may introduce potential biases. These findings underscore AI’s potential as an auxiliary diagnostic tool, necessitating further large-scale prospective validation.