A comprehensive literature search was undertaken using the databases of Embase, MEDLINE, and the Cochrane Library. All published articles up until October 2020 were included. Search terms used in Embase and MEDLINE included “colon*,” “polyp,” “artificial intelligence OR machine learning,” and “computer aided or assisted and diagnos* OR detect*.” Studies in the Cochrane Library were identified with the terms “colonic polyp,” “artificial intelligence,” and “diagnosis, computer-assisted” (Multimedia Appendix 1).

Inclusion criteria were as follows:

Exclusion criteria were as follows:

The retrieved articles were screened for duplicates by two reviewers; these were excluded. Titles and abstracts were then screened for relevance by two reviewers independently, and irrelevant studies were excluded. Following this, full-text reviews of remaining studies were completed. The reference lists of identified review articles and included papers were scrutinized for relevant studies. Disagreements about eligibility were settled by consensus, both after screening and following full‐text review. Inclusion and exclusion criteria were met by all final articles.

Data were gathered from studies and placed onto a standard spreadsheet template. For each study, we extracted the following data: study details (ie, first author, year of publication, and journal), primary outcome (ie, polyp detection vs characterization), study design (ie, type of study, method of AI, and exclusion criteria), information on type of imaging modality (ie, images or videos, images for training, and images for validation), and information regarding diagnostic accuracy characteristics (ie, sensitivity, specificity, accuracy, ADR, and PDR).

Study quality was independently assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool [16]. Each domain was classified as low-risk, high-risk, or unclear risk of bias. For randomized controlled trials (RCTs), the Jadad scale was used for quality scoring [17]. Studies with a Jadad score of 3 or more were considered good quality.

Independent proportions and their differences were calculated and pooled through DerSimonian and Laird random-effects modeling. This considered both between-study and within-study variances, which contributed to study weighting. Pooled values and 95% CIs were computed and represented on forest plots. Statistical heterogeneity was determined by the I² statistic, where <30% was low, 30%-60% was moderate, and >60% was high. Analyses were performed using Stata, version 15 (StataCorp LLC). Probability values of P≤.05 were considered statistically significant.

A total of 899 articles were identified from the database searches. After removing duplicates, 575 records were screened on the basis of titles and abstracts. A total of 141 articles were identified as appropriate for full-text review. Further evaluation and application of the exclusion criteria revealed 48 studies, which were included in this systematic review and meta-analysis. The study screening and selection process is shown in Figure 1.

Studies in this systematic review included preclinical studies for polyp detection (Table 1 [18-35]), preclinical studies for polyp characterization (Table 2 [11,13,36-55]), and recent RCTs (Table 3 [56-63]). The studies were all published between 2003 and 2020. The outcome measures were polyp detection in 18 studies, polyp characterization in 22 studies, and PDR in 8 studies. The studies analyzing sensitivity, specificity, and accuracy when testing each AI system were found to present results at the per-patient, per-polyp, and/or per-image levels, whereas the RCTs evaluating the ADR and PDR consistently presented per-patient results.

Studies for polyp detection predominantly used CNN or DCNN as their machine learning approach. A total of 14 studies for polyp detection were carried out retrospectively. There was a large variation in the number of images used by each paper to train or validate the AI systems in detecting polyps, with one study using 8 images [20] to train the system, while another used 5545 images [25].

In the majority of studies, narrow band imaging (NBI) or endocytoscopy was the imaging method of choice for characterizing polyps, with one exception in which the imaging modality was not stated [47]. Data for polyp characterization was gathered retrospectively in 18 studies. In 3 studies that collected data prospectively, a support vector machine classifier was used as the machine learning approach. Similarly to studies for polyp detection, those analyzing polyp characterization had a large variation in number of images used for training or validating the AI system. However, studies for polyp characterization focused more on the number of polyps used than on overall images, as seen in Table 2.

The diagnostic accuracy of the machine learning systems for detecting polyps was assessed using 103,049 still images in 10 studies, reporting a pooled sensitivity of 0.84 (95% CI 0.74-0.93), a specificity of 0.87 (95% CI 0.83-0.90), and an accuracy of 0.89 (95% CI 0.81-0.97). Lesions within video frames or images were used by 14 studies to report the diagnostic performance of their detection systems, highlighting a sensitivity of 0.92 (95% CI 0.89-0.95), a specificity of 0.89 (95% CI 0.84-0.94; Figure 2), and an accuracy of 0.87 (95% CI 0.76-0.97). There were 11 studies analyzing the accuracy of polyp detection through the use of images or video clips gathered from more than 17,401 patients. These demonstrated a sensitivity of 0.92 (95% CI 0.90-0.94), a specificity of 0.93 (95% CI 0.91-0.96), and accuracy of 0.92 (95% CI 0.87-0.98).

There were 9 studies reporting diagnostic accuracy characteristics for computer analysis of single image frames. These included a total of 22,862 images and demonstrated a sensitivity of 0.92 (95% CI 0.90-0.95; Figure 3), a specificity of 0.79 (95% CI 0.68-0.91), and an accuracy of 0.87 (95% CI 0.83-0.91). A further 20 studies assessed the diagnostic accuracy of techniques for predicting the histological diagnosis of a polyp, with a sensitivity of 0.94 (95% CI 0.92-0.95), a specificity of 0.87 (95% CI 0.83-0.90), and an accuracy of 0.91 (95% CI 0.88-0.93). A total of 16 studies analyzed diagnostic accuracy using images or video clips from a cohort of 4001 patients having undergone colonoscopy. These studies showed a sensitivity of 0.94 (95% CI 0.92-0.95), a specificity of 0.82 (95% CI 0.73-0.91), and an accuracy of 0.90 (95% CI 0.86-0.94).

The 8 RCTs consisted of a total of 5577 patients: 2438 patients in the AI group and 2463 patients in the control group with standard colonoscopy alone [56-59]. These captured data prospectively with the use of deep learning methods on real-time video streams from colonoscopy.

The meta-analysis showed a significant increase in pooled PDR in patients with the use of AI for polyp detection during colonoscopy compared with patients who had standard colonoscopy (odds ratio [OR] 1.75, 95% CI 1.56-1.96; P<.001; Figure 4). The PDR ranged from 38% to 64% when using AI, with a median of 45%. When comparing patients undergoing colonoscopy with the use of AI to those having standard colonoscopy, there was also a significant increase in ADR (OR 1.53, 95% CI 1.32-1.77; P<.001; Figure 5). The ADR ranged from 16% to 55% with a median of 34% when using AI technology compared to standard colonoscopy.

There was a high degree of variation between studies. The heterogeneity was statistically significant when comparing the studies for polyp detection and characterization and assessing for sensitivity, specificity, and accuracy (P<.05). The lowest variation for polyp detection was among the studies assessing accuracy with polyp data (I²=86.3%), and the highest was among those analyzing the sensitivity of machine learning systems using image data sets (I²=99.9%). When considering studies for polyp characterization, the heterogeneity was lowest for studies analyzing sensitivity using patient data sets (I²=51.1%) and highest when assessing specificity using image data sets (I²=99.9%). Within the RCTs assessed, there was found to be a low degree of heterogeneity for PDR (I²=0%; P=.70) and a moderate degree of heterogeneity for ADR (I²=45.5%; P=.09). These results were not statistically significant.

The assessment of bias for the studies when using the QUADAS-2 tool is depicted in Table S1 in Multimedia Appendix 2. Most of the RCTs scored 3 or more on the Jadad scale and were, therefore, considered to be of good quality (Table S2 in Multimedia Appendix 2). One study scored 2, suggesting poor quality, but after reviewing the paper and its evidence in detail, the paper was included in the final analysis [64]. This is because despite the lack of mention of blinding, the selection process for participants was justified with consecutive patients enrolled, and there were no concerns regarding applicability. The paper matched the selection criteria of our study and was otherwise in line with other studies that were included.

The aim of this systematic review and meta-analysis was to examine the current status of diagnostic accuracy for AI-based technologies in the detection and characterization of colorectal polyps. We found a wide variety of machine learning systems being used for polyp detection and characterization in numerous studies. The overall diagnostic accuracy for these systems to detect polyps was high, predominantly with sensitivities, specificities, and accuracies above 84%. When characterizing polyps, the majority of machine learning systems had sensitivities, specificities, and accuracies above 82%. These outcomes show good results for current machine learning systems and algorithms to detect and characterize polyps, and indirectly in regard to the rate of false positives.

This meta-analysis highlights a significant increase in PDR and ADR when using AI systems in conjunction with colonoscopy in real time to detect polyps in the colon and rectum with an overall OR of 1.75 (95% CI 1.56-1.96; P<.05) and 1.53 (95% CI 1.32-1.77; P<.05), respectively. The UK key performance indicators and quality assurance standards for colonoscopy dictate that the minimal ADR should be 15%, with an aspirational target of 20% [65]. It has previously been shown that endoscopists with an ADR of less than 20% had a hazard ratio for interval cancer that was 10 times higher than those with an ADR of greater than 20% [66]. All RCTs in this review were shown to have an ADR of greater than 15% when detecting polyps with the use of an AI system, the majority of which highlighted an ADR of greater than 25% [56-58]. These outcomes are a promising start for the use of AI to detect missed polyps and, thus, may lead to a reduction in CRC incidence.

The assessment of quality of the diagnostic accuracy studies included in this paper highlighted an overall low risk of bias, justifying the validity of the study results and implying that their results may be applicable to clinical practice. The main area of bias in the RCTs was in the process of blinding. This may have contributed to an overestimation in the effects of AI in polyp detection.

There are many limitations within the published studies (Table S1 in Multimedia Appendix 3). Factors contributing to the miss rate of polyps are multifactorial and include patient-related factors, polyp-related factors, and image-related factors [67,68]. It is encouraging to note that a variety of imaging modalities were used in the studies in this review, since this will improve applicability in a clinical setting. We note that most studies with image enhancement techniques have used NBI, and it will be important to validate the performance of AI systems in endoscopy using image enhancement approaches from other manufacturers (eg, i-scan from PENTAX Medical and blue laser imaging from Fujifilm Corporation). Some studies analyzing polyp characterization used magnification NBI [11,36,38,69]. This imaging modality is not commonly used in Western endoscopic practice, so is less applicable to a health care setting in the Western world. Although there has been significant development in computer-assisted technologies to increase ADR, issues with image quality still remain. Many studies in this review excluded images that were blurred or of poor quality when assessing diagnostic accuracy of the machine learning systems. [27,40,42,51]. Recent RCTs have tried to tackle this problem by developing models to recognize blurry frames [58,59]. Other studies excluded images with poor bowel preparation [27,36,48]. Adequate bowel cleansing is vital for complete mucosal inspection; however, it has been shown in a meta-analysis that low-quality preparation does not significantly affect ADR, since these patients frequently undergo repeat colonoscopy [70]. Most RCTs included in this review used the Boston Bowel Preparation Scale [71] to assess adequacy of bowel preparation.

Sufficient withdrawal time allows full mucosal inspection with careful examination of all folds and flexures, in an attempt to avoid missing any polyps. It has been shown that an increase in withdrawal time is associated with an increase in ADR [72]. This supports the use of withdrawal times as a quality indicator for screening colonoscopy. In preclinical studies, it is difficult to assess withdrawal times given the use of still images and video clips. In the RCTs assessed, the withdrawal times—excluding biopsy time—were mostly higher with the use of AI-based technology, although not significantly so in all studies (Table 3). However, the ability to record the withdrawal time is equally important [58,59]. This may suggest that quality control during colonoscopy examinations can be maintained with the use of machine learning.

Given the fact that AI is a relatively new and evolving area of medical practice, there is a lack of evidence-based standards to support its development. This is highlighted through the inconsistencies in validating the machine learning systems in each study. The data used for training the algorithms vary in type, for example, as either a static image from the colonoscopy [45,46] or an image of a polyp [21,47], and in number, with some studies having very small sample sizes [21,52]. We acknowledge the high degree of heterogeneity in the included studies, which may, in part, be explained by the wide range of approaches or algorithms used. This may suggest that our findings are applicable to a wide range of study settings and outcomes. However, the high degree of heterogeneity also emphasizes the issue of inconsistencies within the development of AI systems and, thus, weakens their design and may hinder implementation of the AI systems in a clinical setting. In order to address this problem, we are developing a new multidisciplinary, consensus-based reporting standards statement called STARD-AI (Standards for Reporting of Diagnostic Accuracy Studies–Artificial Intelligence). It is being developed to provide stringent guidelines for all AI-based clinical trials that report diagnostic accuracy [73,74].

The lack of standards among these studies introduces an element of selection bias. In traditional computer programming, intelligent systems were built by writing models by hand and, therefore, understanding the rules from which conclusions were made. Neural networks and deep learning techniques are criticized for their “black box” problem, in failing to produce an intelligible description of the results produced. This creates tension between our need for explanations and our interests in efficiency. Most studies in this systematic review did not reveal their algorithms, which begs one to question whether they only used the algorithms that were most successful in producing the desired outcome without understanding the process underlying it.

Multiple other factors contribute to the lack of applicability of these studies in clinical practice. Many of the studies about polyp detection and characterization have been carried out in Japan [46,50,51] or China [19,56,59], and differences in polyp biology and tumorigenesis may limit application to Western endoscopy practice [75]. Furthermore, for real-time detection to be successful, the operation of the AI system to detect and characterize polyps must be fast, practical, and nondisruptive to workflow. However, most current studies are designed in a nonclinical environment and carried out retrospectively, with only a handful of recent RCTs. More RCTs are needed to provide prospective data by testing the machine learning systems while a colonoscopy procedure is undertaken.

The financial implications of introducing an AI system to endoscopy should be considered. The studies in this review lack evidence to show that AI systems would be cost-effective. Before clinical application, studies must demonstrate that the current burden on health care systems and histopathology departments can be relieved, both in view of workload and in terms of costs. A very recent study examining the use of AI combined with the diagnose-and-leave strategy for diminutive polyps has found substantial reductions in the cost of colonoscopy based on prospective data [76]. This is an encouraging outcome, but more studies are needed.

The role of the health care workforce must also be considered in a time of developing AI systems. At present, real-time detection systems during colonoscopy are not able to operate independently of human direction, but understanding the change in the role of the endoscopist and nurses will be crucial for the future. In addition, a skills gap to prepare the workforce for AI will need to be addressed. The refinement of machine learning systems in detecting polyps will eventually lead to the use of AI in conjunction with all routine colonoscopy procedures. This will allow the procedure to be performed by staff who will not require the lengthy training or accreditation [77]. In this scenario, only patients with complex polyps requiring more advanced management may need to be referred to expert endoscopists.

It is important to also consider some of the ethical dilemmas that arise from the use of AI in health care. The aim of AI in polyp detection and characterization is to introduce machine learning as a “checker system” for the endoscopist. As a result, incorporation of AI into endoscopy should be encouraged as a complementary tool and not as a replacement for a clinician. For this reason, a high degree of accuracy is required from AI systems. We expect that they operate with 100% sensitivity and a low rate of false positives. However, AI is not yet free from bias or errors, and an AI decision support tool could easily succumb to automation bias when its predictions are almost always followed by the endoscopist [78]. Machine learning systems can also unintentionally reproduce or magnify existing biases of their training data sets and exacerbate health disparities [79]. Many of the studies in this meta-analysis, for example, have excluded patients with IBD or sessile serrated polyps [39,43,56], limiting their applicability for these populations. We recognize that these other cohorts of patients, including those with benign colonic pathologies and not exclusively polyps, are important to include in such research. However, this technology is still in its infancy and these patient groups represent a minority. It is difficult and not entirely feasible to create validated AI algorithms for all patient cohorts until the technology is more established and works well in its own right.

Although this systematic review has shown the performance of the AI systems to be satisfactory, the majority of the studies are preclinical trials that have not addressed these clinical needs. As a result, there remains a lack of confidence by endoscopists and patients to fully adopt the system as a whole. The clinical expectations exceed the aims of the machine learning algorithms. To fully support the incorporation of an AI system into routine practice, the diagnostic accuracy for polyp detection and characterization must meet the desired threshold, while also providing confidence that quality requirements will be fulfilled.

A further two challenges threaten the ability for AI to thrive in health care: patient confidentiality and accountability. The lack of stringent policies for the use of training data in AI means that the methods used to deidentify patient information are weak, and we suggest that standardized guidance is required for the consent of collection and use of patient data for AI training purposes. Once an algorithm-based health care system is operational, the question of accountability arises. In the case that a machine learning system working in unison with an endoscopist detects and characterizes a polyp as hyperplastic when, in fact, it is adenomatous, who is held liable for this mistake? A robust legal framework in association with national and international endoscopy representative groups (eg, the Joint Advisory Group on Gastrointestinal Endoscopy in the United Kingdom and the ASGE in the United States) for the use of AI in health care is vital to protect endoscopists and patients. Addressing these important concerns will help build confidence and trust among patients and doctors for the use of machine learning in the delivery of care.

This systematic review and meta-analysis highlights the growing interest in the field of polyp detection and characterization during colonoscopy using AI. The current accuracy of machine learning for this role is high. There is potential to improve ADR and, consequently, reduce the incidence of CRC.

However, AI and machine learning systems are still evolving. Firstly, higher-quality research with modern trial designs is needed in this field, with particular attention on using larger data sets and by validating the AI systems prospectively in a clinical setting. Secondly, these systems must provide quality assurance with a robust ethical and legal framework before they can be fully embraced by clinicians and patients in the future.