Early gastric cancer was defined as mucosal and submucosal (T1) gastric cancer irrespective of lymph node involvement. Studies involving advanced gastric cancer, precancerous lesions such as intestinal metaplasia and dysplasia, and gastric cancer without specific annotations were excluded. The accuracy of AI was defined as the area under the hierarchical summary receiver operating characteristic curve or the area under the curve (AUC).

This meta-analysis was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We systematically searched the PubMed, Embase, Cochrane Library, and Web of Science databases for studies that assessed the diagnostic accuracy of AI in early gastric cancer from endoscopic images from database inception to June 2020. We used “gastric cancer,” “endoscopy,” and “artificial intelligence” as relevant terms with Boolean operators “OR” and “AND” (Multimedia Appendix 1). Two authors, P-CC and L-YR, independently screened the study titles and abstracts. Studies that used AI to diagnose early gastric cancer from endoscopic images were included. Studies that did not provide a 2×2 contingency table were not included in the final analysis. This study was registered in PROSPERO (registration CRD42020193223).

The quality of the included studies was assessed independently by 2 authors (P-CC and L-YR) on the basis of the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2), and all disagreement was resolved through discussion with the third author (Y-NK). The assessment included risk of bias and applicability to the QUADAS-2 domains: patient selection, index test, reference standard, and flow and timing. From the included studies, we extracted data on the number of endoscopic images of lesions diagnosed as early gastric cancer (ie, true positive), the number of endoscopic images of benign lesions misdiagnosed as malignant (ie, false positive), the number of endoscopic images of malignant lesions misdiagnosed as benign (ie, false negative), and the number of endoscopic images of benign lesions correctly diagnosed as benign (ie, true negative). We also extracted data on the country of origin, AI methods, and image modalities used.

The primary outcome was the accuracy of AI to diagnose early gastric cancer from endoscopic images. Secondary outcomes focused on the sensitivity analysis of (a) different AI methods, (b) endoscopic imaging modalities, (c) studies that compared AI and endoscopist performance, (d) studies that evaluated larger gastric lesions (>20 mm), (e) studies that simply differentiated abnormal and normal lesions rather than using pathological staging, and (f) studies that separated the training and testing data sets during AI training. Sensitivity analysis was conducted if a subgroup contained more than two studies. We only assessed the heterogeneity of the included studies. Following extraction, the data were primarily analyzed using STATA 14 (StataCorp LP, StataCorp) except for subgroups with fewer than four studies. The midas and metandi commands were used to determine sensitivity, specificity, and AUC and analyze the summary receiver operating characteristic (SROC) and hierarchical summary receiver operating characteristic (HSROC) curves. Basic formulas for the analyses were as follows:

In the formulas, “a” is the intercept, “b” is slope, and DOR refers to the diagnostic odds ratio. Moreover, TPR is the true positive rate, and FPR is the false-positive rate. The modchk tool was used to examine goodness-of-fit and bivariate normality before SROC analysis in a bivariate mixed-effects model. The metabias command and the pubbias syntax were used to perform the Egger test and Deeks funnel plot asymmetry tests, respectively. The Egger test for diagnostic meta-analysis was based on the formula proposed by Hasselblad and Hedges, and the formula is mainly to detect publication bias detection via testing standard normal deviate among the included studies [32,33].

In the regression model, with intercept “a” and slope “b,” the standard normal deviation could be estimated by using diagnostic d divided by SE of the diagnostic d. The metaprop package in STATA was mainly used to synthesize the sensitivity and specificity. I² statistics were used to determine levels of heterogeneity via the formula as follows:

where Q refers to Cochran Q, and df is the degree of freedom. Because R software (The R Foundation) does not restrict the number of observations used in the meta-analysis, it was used for sensitivity analysis if subgroups consisted of fewer than four studies. Indeed, a meta-analysis in R could be carried out when more than two studies report the same outcome by pooling data with logit transformation and Clopper-Pearson interval method (also called exact binomial interval) based on inverse variance. Function metaprop in package meta for R was applied to carry out sensitivity analysis, and the mada package in R was used to calculate the pooled accuracy. Besides, the metagen package in R was used to synthesize endoscopist performance because of the lack of detailed data on each endoscopist.

Of the 5591 studies identified in the literature review, 5265 underwent title and abstract screening after duplication removal. The flowchart of the literature review process was constructed according to the PRISMA flowchart format (Figure 1). We excluded 5132 irrelevant studies and assessed the eligibility of the remaining 133 studies through full-text reading. Studies evaluating nonearly gastric cancer (eg, advanced gastric cancer and metaplasia) were excluded. Overall, 23 studies investigated the performance of AI on early gastric cancer diagnosis from endoscopic images. Finally, 12 studies comprising a total of 11,685 cases were included in the meta-analysis [34-45].

Detailed information on the 12 studies is listed in Table 1. All studies were conducted in Asia, including Japan (k=8), China (k=2), and Korea (k=2), in or after 2012. All were case control studies with testing data sets containing 81 to 3390 images. Patients in 10 studies had pathological proof of early gastric cancer, whereas in the other 2 studies, the endoscopic images were collected through description. White light imaging (WLI), narrow-band imaging (NBI), flexible spectral imaging color enhancement, and mixed imaging modalities were used in 4 (33%), 2 (17), 1 (8%), and 2 (17%) studies, respectively. Moreover, 8 (67%) studies used deep learning methods (eg, convolutional neural networks) as their AI backbone, and 3 (25%) studies employed nondeep learning methods (support vector machines and discriminant analysis of principal components). Comparisons of the diagnostic performance of AI and endoscopists were conducted in 3 (25%) studies, and 2 (17%) studies included endoscopic images of small lesions (<20 mm) in early gastric cancer. In 3 (25%) studies, the training and testing data sets were not separated for AI training. Table 1 presents a detailed description of the 12 studies.

We also assessed the quality of the studies along with the risk of bias according to the revised QUADAS-2 tool (Multimedia Appendix 2). All studies, including the 3 that failed to separate the training and testing data sets, had high bias risks for patient selection because of their retrospective design. Moreover, 2 (17%) studies assessed early gastric cancer but did not mention pathological staging. Thus, they were classified as having a high risk of bias for the index test.

To assess the diagnostic ability of AI to detect early gastric cancer from endoscopic images, we performed a meta-analysis on the selected 12 studies. Goodness-of-fit (Figure 2A) and bivariate normality (Figure 2B) demonstrated that the included data were appropriate for further analysis. The pooled sensitivity and specificity of AI were 0.86 (95% CI 0.75-0.92) and 0.90 (95% CI 0.84-0.93), respectively (Figures 2C and 2D). Empirical Bayesian predictions were consistent with the observed sensitivity and specificity (Multimedia Appendix 3). Highly heterogeneous estimates (I²>90%) necessitated subgroup analysis and sensitivity analysis. Laminated figures of the SROC and HSROC plots indicate an AUC of 0.94 (95% CI 0.92-0.96) with a confidence region (Figure 3A). However, the scatter matrix (Multimedia Appendix 4) suggests that in clinical practice, diagnosis of early gastric cancer may not substantially benefit from AI assistance. The Deeks funnel plot asymmetry test (Figure 3B) and Egger test (Multimedia Appendix 5) did not detect significant publication bias in the pooled results of AI-assisted diagnosis of early gastric cancer.

We assessed the diagnostic performance of various AI methods and endoscopic imaging modalities for early gastric cancer (Table 2). The pooled sensitivity and specificity in studies using deep learning methods were 0.84 (95% CI 0.69-0.93) and 0.88 (95% CI 0.80-0.93), respectively. Studies using nondeep learning methods had a pooled sensitivity and specificity of 0.91 (95% CI 0.86-0.95) and 0.90 (95% CI 0.87-0.93), respectively. The accuracy of the nondeep learning group (AUC=0.96) was higher than that of the deep learning group (AUC=0.93; Multimedia Appendices 6 and 7).

For endoscopic imaging modalities, studies using WLI had a sensitivity and specificity of 0.73 (95% CI 0.42-0.91) and 0.89 (95% CI 0.76-0.96), respectively. Studies using NBI reported a sensitivity and specificity of 0.96 (95% CI 0.92-0.98) and 0.83 (95% CI 0.54-0.95), respectively. The accuracy of the NBI group (AUC=0.96) was higher than that of the WLI group (AUC=0.90; Multimedia Appendices 8 and 9). Table S1 (Multimedia Appendix 10) shows a comparison of the diagnostic performance of AI and endoscopists for early gastric cancer from the three studies (n=91).

We excluded some studies with a high risk of bias and performed sensitivity analysis on the remaining studies (Tables S2-S5 Multimedia Appendices 11-14). Furthermore, we also examined how the results were affected by studies with unknown AI methods. Sensitivity analyses indicated that pooled estimates were not seriously affected by the factors (Table 2). Lower heterogeneity and specificity were observed in endoscopist performance when we excluded studies that only evaluated small lesions and studies that predicted pathological staging (Tables S2 and S3 in Multimedia Appendices 11 and 12). Lower heterogeneity was also noted in WLI subgroups if the training and testing data sets were separated for AI training (Table S4 in Multimedia Appendix 13). No other additional analyses provided credible evidence.

To our knowledge, this was the first systematic review and meta-analysis of AI-assisted endoscopic diagnosis of early gastric cancer. The accuracy, sensitivity, and specificity were 0.94, 0.86, and 0.90, respectively. High heterogeneity was noted. Sensitivity analysis revealed less heterogeneity in studies using nondeep learning AI methods and endoscopic NBI.

Our results indicate good sensitivity and specificity of AI-assisted detection of early gastric cancer. However, high heterogeneity was also noted among the included studies, which may be attributed to between-study differences in machine learning methods and imaging modalities [46]. In a meta-analysis of AI prediction of colonic polyp histology, AI performance was better when deep learning was used as a backbone and when NBI was used to identify the lesions [46]. In this study, we also investigated the roles of various machine learning methods and imaging modalities. Unfortunately, only 2 studies in the deep learning subgroup used the same deep learning algorithm, and no two studies in the nondeep learning subgroup classified the lesions according to the same features. Only 6 studies specified their endoscopic imaging modalities. Less heterogeneity was observed in the nondeep learning and NBI groups, possibly because of the compliance of early gastric cancer diagnosis to the vessel plus surface classification system under NBI. This indicates that nondeep learning methods and NBI may provide more consistent results and can be applied in clinical practice earlier than deep learning methods and WLI. Further investigations are warranted.

We assessed the diagnostic performance of AI and endoscopists (n=91) for early gastric cancer detection, which was compared in 3 studies. The endoscopists were assigned to only 1 subgroup because of the inconsistent definitions of expert and nonexpert endoscopists between studies. The sensitivity and specificity of AI were 0.67 and 0.87, respectively, and those of the endoscopists were 0.68 and 0.92, respectively. In both groups, diagnostic performance varied widely with high heterogeneity. The diagnostic performance of AI was better than that of WLI compared with other studies; a meta-analysis reported a pooled sensitivity and specificity of 48% and 67% between endoscopists and WLI, whereas those between endoscopists and NBI were 83% and 97%, respectively [47]. In this study, AI and endoscopist performance were comparable in individual studies, but this effect diminished when studies were pooled. Further research comparing AI and endoscopist performance for early gastric cancer diagnosis is required.

Only 2 of the included studies evaluated only small lesions [37,40]. Smaller lesions and mucosal lesions were less accurately detected by AI [42]. Kanesaka et al [37] included only depressed and small (<10 mm) lesions, and the AI system of nondeep learning methods was trained using a small data set of 126 images from NBI. In another study, early gastric cancer lesions less than 20 mm in diameter were included in the WLI testing data set, and the deep learning AI system was trained using a data set of 13,584 images of early and advanced gastric cancer [40]. Because these 2 studies used distinct materials and methods, their findings may not be representative. The accuracy of AI-assisted detection of small gastric cancer lesions warrants further investigation.

Some studies have explored the application of AI to other aspects of gastroendoscopy. For example, Wu et al [39] used AI to monitor endoscopic blind spots and identify regions indicative of early gastric cancer. A randomized controlled trial in China reported that AI reduced the rate of endoscopic blind spots [48]. Other studies have tested the accuracy of AI in predicting the invasion depth of gastric cancer—conventionally assessed through endoscopic ultrasound—from endoscopic images. In their study of AI-assisted simultaneous detection of gastric cancer and invasion depth, Yoon et al [42] reported a sensitivity and specificity of invasion depth of 79.2% and 77.8%, respectively. In a study by Zhu et al [31], the predicted sensitivity and specificity from the T1 to the T4 stage were 76% and 96%, respectively. Nevertheless, relevant evidence is limited, and further investigation is required.

The considerable advancement of AI in precise image recognition challenges the roles of physicians in disease diagnosis. AI systems offer certain advantages over physician diagnosis, the foremost of which are faster image processing rates and continuous work. In all included studies that specified image processing time, that of AI systems was shorter than that of endoscopists. AI assistance may reduce the risk of human error that arises from performing numerous endoscopic examinations. Moreover, the training of AI systems is considerably faster and less complicated than that of endoscopists. Well-trained AI systems learn from analyzing numerous images, whereas endoscopists rely on their individual skills and clinical experience. Training endoscopists is expensive and time-consuming because of the steep learning curve for the various image-enhancing techniques. In addition, AI may work as a double-check system during or after endoscopy, given its high sensitivity and specificity. AI allows for a second opinion, which is particularly valuable now that gastroendoscopy has been popularized and nationwide screening for gastric cancer has been implemented.

Our study had several limitations. First, all the included studies were retrospective case control studies performed in Asia, some of which compared early gastric cancer and normal gastric tissues, and some compared benign gastric lesions such as ulcers and gastritis. The possibility of selection bias cannot be ruled out. A randomized controlled trial comparing the diagnostic performance of AI and endoscopists for early and advanced gastric cancer (NCT04040374) is currently underway. Second, all the studies identified gastric lesions from still, clear, endoscopic images; images with blood or mucus were excluded. In daily practice, however, gastroendoscopy is recorded in video format, and still images are only captured for suspicious lesions. Blood, food debris, mucus, and foam, which reduce the accuracy of AI, are commonly encountered during examination [39]. Several studies have reported excellent accuracy of AI systems in recognizing gastric cancer from endoscopic video [39,49]. However, further studies and faster image processing rates are necessary. Third, our pooled estimates were highly heterogeneous, and the subgroup and sensitivity analyses did not substantially reduce heterogeneity. The statistical heterogeneity may be ascribed to differences in the AI methods and endoscopic imaging techniques. These potential sources of heterogeneity should be discussed in future research. At present, AI may assist endoscopists in double-checking suspicious lesions.

To our knowledge, this is the first meta-analysis of the performance of AI in detecting early gastric cancer using endoscopic images. The available evidence suggests that AI can support the diagnosis of early gastric cancer; however, the collocation of imaging techniques and optimal algorithm remains unclear. Larger prospective cohort studies should be conducted to further validate the diagnostic accuracy of AI. Moreover, competing models of AI for the detection of early gastric cancer are worthy of future investigation.