This scoping review was reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines [20] and conducted according to the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy [21]. We adopted a scoping review approach [22,23] because of a broad set of inclusion criteria. The protocol for this review was published in BMJ Open [24]. We were unable to register this protocol with PROSPERO as it does not include scoping reviews.

We performed a librarian-assisted search on the Medical Literature Analysis and Retrieval System Online (MEDLINE) (Ovid), EMBASE (Ovid), and the Cochrane Library for papers published from January 2000 to November 2018. Articles published before 2000 were excluded because before that smartphone technology was limited. We used both medical subject headings (MeSH) and keywords relating to DR (eg, “diabetic retinopathy,” “macular edema,” and “diabetic maculopathy”) and to smartphones (eg, “mobile health,” “mobile phones,” and “applications”) or AI (eg, “artificial intelligence” and “machine learning”; Multimedia Appendix 1). We also explored the bibliography of both primary articles and reviews to identify potentially eligible studies missed by the electronic search.

The inclusion criteria were as follows: (1) studies evaluating the diagnostic test accuracy of smartphone ophthalmoscopy for detecting DR in patients with type 1 or 2 DM; (2) studies utilizing a smartphone’s in-built camera for retinal imaging, including the use of any attachments externally fitted to the smartphone; (3) studies comparing smartphone ophthalmoscopy with any acceptable and commonly employed reference standard, such as fundus photography, indirect ophthalmoscopy, slit-lamp biomicroscopy, or fluorescein angiography; (4) studies employing any kind of health care professional to acquire the smartphone images. Language was not an exclusion criterion.

Examples of eligible smartphone ophthalmoscopy techniques include the following:

Covidence software (Veritas Health Innovation, Melbourne, Australia) was used to remove duplicated studies [25]. After a pilot screening of 20 citations to calibrate the judging criteria, two reviewers independently screened all articles retrieved from the search strategy by title and abstract, using Covidence. Subsequently, we screened the full text of the remaining articles and performed data extraction using a prepiloted form. Any disagreements were resolved through consensus.

A data extraction form (Multimedia Appendix 2) was created and piloted to record the following data from each study: (1) study author and date published; (2) sample size; (3) participant characteristics (eg, age, duration and type of DM); (4) information regarding imaging techniques (eg, details about smartphones and adaptors used, image resolution); (5) health care professional performing smartphone ophthalmoscopy; (6) reference standard used; and (7) test results (eg, true positives [TP], false positives [FP], true negatives [TN], and false negatives [FN]). Corresponding authors were contacted for additional details or missing data required to construct a 2×2 table. Two reviewers independently extracted study data using a data extraction template created in Microsoft Excel, with disagreements resolved via consensus.

The Quality Assessment of Diagnostic Accuracy Studies tool, QUADAS-2, consisting of descriptions and signaling questions, was used to assess the risk of bias and applicability of all included studies in four domains pertaining to (1) patient selection, (2) index test, (3) reference standard, and (4) flow of participants through the study and timing between the index test and reference standard [26]. Two reviewers independently assessed study quality, and disagreements were resolved via discussion until a consensus was reached.

We constructed 2×2 tables based on data from each study. The sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR−), diagnostic odds ratio (DOR), and area under summary receiver operating characteristic (SROC) curve were calculated using a random effects model because of the high expected heterogeneity [27]. We constructed SROC using the bivariate model where possible. Being both a hierarchical and random effects model, the bivariate model is preferred to the Moses-Littenberg SROC curve—the former method accounts for between-study heterogeneity. For SROC curves employing the bivariate model, elliptical 95% confidence regions were obtained by joining the individual confidence regions for logit sensitivity and logit specificity via parametric representations [28,29].

Heterogeneity was evaluated using chi-square (χ²) and I² values of likelihood ratio tests (LRT) or DOR, with I²<25%, 25–75%, and >75% representing low, moderate, and high degree of inconsistency, respectively. Threshold effect was measured using the Spearman correlation coefficient ρ between logits of sensitivity and specificity, with ρ closer to −1 indicating higher threshold effect and better fit of the SROC curve. If information regarding a condition’s prevalence was available from the literature, we calculated the posttest probability using the Fagan nomogram. A P<.05 was considered statistically significant. All analyses were performed using Review Manager version 5.3 from the Cochrane Collaboration [30], METANDI and MIDAS commands in Stata 15.1 (StataCorp, College Station, Texas), Meta-Disc version 1.4 (Ramón y Cajal Hospital, Madrid, Spain) [31], and mada package in R.

Our search strategy yielded 1571 unique records. Of those records, the full text for 41 articles was assessed, and nine studies [32-40] met the inclusion criteria (Figure 1). Two [32,33] of the included studies were conference abstracts. A total of 1430 diabetic patients (at least 2743 eyes) were recruited among these studies.

All studies reported smartphone fundoscopy techniques involving mydriatic, color, and nonstereoscopic imaging (Tables 1 and 2, Multimedia Appendix 3). A total of four studies originated from India, three studies from the United States, and one from Italy. All studies which reported data on gender recruited both males and females. These fundus photographs were graded by ophthalmologists, retinal specialists, or artificial intelligence (AI). In all, seven studies utilized direct ophthalmoscopy to acquire smartphone fundus images, while two studies [38,40] used indirect ophthalmoscopy.

A total of five studies [32,33,37,39,40] employed slit-lamp biomicroscopy as the reference standard, of which two studies complemented the examination with dilated indirect ophthalmoscopy [39,40]. In all, two studies [35,38] utilized 7-field mydriatic fundus photography using a tabletop fundus camera; one study [34] used traditional in-clinic diagnosis including a dilated eye examination; and one study [36] utilized ophthalmologists’ grading of the same smartphone-acquired images as the reference standard. Overall, four studies [32,36,37,40] utilized the International Clinical DR Disease Severity Scale to grade DR; three studies [34,35,38] employed the Airlie House or modified ETDRS criteria; and one study [39] used the United Kingdom’s National Health Service (NHS) guidelines. Referral-warranted DR (RWDR) was defined as moderate NPDR or worse or DME; vision-threatening DR (VTDR) as severe NPDR, PDR, or DME; and sight-threatening DR (STDR) as PDR or DME. Health professionals performing smartphone ophthalmoscopy included medical students, interns or assistants, retinal specialists, ophthalmologists, and ophthalmic photographers. Most studies reported no funding sources or conflicting interests, while such information was unavailable for two studies [32,33]. In all, two studies [34,40] received funding, one of which disclosed multiple authors holding positions in DigiSight Technologies, Inc.

We carried out the quality assessment of the included studies using the QUADAS-2 criteria (Figure 2, Multimedia Appendix 4). Most studies were of high quality with low risk of bias and applicability concerns. A total of four studies had an unclear risk of bias for patient selection because of the lack of information regarding patient sampling or inappropriate exclusions. The two abstracts were of lower quality than the other studies because of the limited amount of information available. One study contained applicability concerns because it employed ophthalmologists’ grading of smartphone fundoscopy images as the reference standard; it was excluded from the meta-analysis.

Overall, smartphone ophthalmoscopy performed well in detecting DR. Depending on the severity of DR, smartphone ophthalmoscopy had different accuracy. Progressing from mild NPDR to PDR, we observed an increasing trend in smartphone ophthalmoscopy’s sensitivity, specificity, and DOR. In addition, smartphone ophthalmoscopy had the best performance in detecting PDR, RWDR, VTDR, and STDR; these are important categories to detect as they can significantly affect vision. The lowest sensitivity was observed for detecting mild NPDR, mainly caused by one study enrolling only 7 participants with RWDR. The DOR was lowest for AI’s detection of RWDR. There was also a low percentage of ungradable images across most studies, implying that smartphone ophthalmoscopy is relatively reliable. Common causes of ungradable images included cataracts, poor pupil dilation, vitreous hemorrhages, or poor image focus.

Most studies performed smartphone direct ophthalmoscopy utilizing one of four different attachments. The included studies also assessed two methods of indirect ophthalmoscopy. Smartphone ophthalmoscopy in the included studies surpasses the UK NHS targets requiring DR retinal imaging equipment to have a minimum resolution of 6 megapixels or 30 pixels per retinal degree [34,37]. Two studies [35,36] assigned DR grades at a patient level instead of assessing each eye individually. In some cases, smartphone apps were used to digitally stitch the multiple images obtained per eye or enhance the image acquisition process by facilitating ergonomic focusing and capturing of images. Videography was used in two studies [37,38]. Different grading criteria and reference standards were applied across the included studies. High heterogeneity among studies was observed for most types of DR—especially for moderate NPDR and PDR—except for studies employing AI to detect RWDR. In other studies reporting on the use of non-AI smartphone ophthalmoscopy in mild NPDR, RWDR only, and RWDR, VTDR, and STDR, a significant proportion of heterogeneity can be attributed to the threshold effect.

The diagnostic accuracy of AI in grading smartphone ophthalmoscopy images was unexpectedly low. In two studies, the specificity and DOR of AI in detecting RWDR was lower than that of human graders (retinal specialists and ophthalmologists). Nevertheless, one of those studies employed both human and AI to grade identical smartphone-acquired images; the specificity of AI was higher than that of humans. In contrast, a 2015 study demonstrated that AI detects RWDR in smartphone ophthalmoscopy images with 100% sensitivity and 80% specificity (AUC 0.94) [41]. In addition, a recent review revealed that AI software achieved high sensitivity and specificity for detecting DR in datasets of fundus images acquired from other imaging modalities [42]. Finally, IDx-DR was the first commercially approved AI-based autonomous diagnostic system for DR detection. In a prospective study of 900 participants, this system attained high sensitivity and specificity of 87.2% (95% CI 81.8%-91.2%) and 90.7% (95% CI 88.3%-92.7%), respectively, in detecting more than mild DR [43].

Smartphone ophthalmoscopy is a safe means of acquiring retinal images [44]. One study [34] surveyed patients on their comfort levels while undergoing retinal imaging and revealed that all participants felt more comfortable with the light from Cellscope Retina than the light from slit lamps. Other studies [38,39] employing either an intrinsic smartphone light source or external light sources reported lower luminance than conventional fundus cameras.

To our knowledge, this is the first meta-analysis evaluating the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR in diabetic patients. A meta-analysis [45] evaluated the agreement between smartphone retinal imaging and retinal cameras encompassing multiple eye pathologies such as DR, glaucoma, and ocular hypertension. It reported excellent image quality in 84.7% of smartphone images, with good diagnostic accuracy; combined κ agreement was 77.8% (95% CI 70.34%-83.70%), AUC=0.86. However, the patient selection was not limited to diabetic individuals. A large study [46] involving 1460 participants (2920 eyes) had previously evaluated the diagnostic accuracy of smartphone ophthalmoscopy for optic disc imaging. Videography was performed with Peek Retina adaptor attached to an 8.0-megapixel Samsung SIII smartphone. This technique demonstrated excellent agreement (weighted κ=0.69) with a reference standard tabletop fundus camera in measuring vertical cup-disc ratio. Using smartphone ophthalmoscopy, 79.5% of eyes were gradable, compared with 86.4% for tabletop retinal imaging. Furthermore, there was no significant difference between image quality acquired by professional and inexperienced photographers. This study reported a lower percentage of gradable eyes compared with most studies included in our scoping review. This could be attributed to inherent differences in the process of DR grading (which requires examination of the retina in general) compared with measuring cup-disc ratio (which specifically examines the optic disc). Regardless of sample size, the agreement of smartphone ophthalmoscopy with a well-established reference standard remains high.

This scoping review aimed to provide a comprehensive analysis of the available literature in this field. Correspondingly, we had broad inclusion criteria encompassing different smartphone ophthalmoscopy techniques, reference standards, DR severity scales, and health care professionals. Smartphone retinal imaging is an emerging technology, and we wanted to capture as much of the available evidence as possible (Multimedia Appendix 5). The included studies were published relatively recently, from 2015 to 2018, heralding future breakthroughs in the diagnostic accuracy of smartphone retinal imaging as affordable and accessible means of DR detection.

Our study employed a comprehensive search strategy and examined studies from different countries involving different types of diabetic patients. At least two reviewers performed quality assessment and data extraction independently following Cochrane methodology. Based on the QUADAS-2 tool, most studies possessed minimal risk of bias and little applicability concerns. In particular, all included studies blinded or masked the graders.

Although the protocol for this scoping review was published in BMJ Open, this protocol was not registered. Three studies [33,34,39] utilized two graders, thereby creating two separate 2×2 tables; to avoid double-counting, we averaged the TP, FP, TN, and FN values, and rounded those average values to the nearest whole number for analysis. For one study [39], we assumed none of the four excluded eyes had DME. Owing to the small number of studies and limited information available, we were not able to conduct a meta-regression analysis or assess for publication bias.

The large 95% CIs for most SROC curves indicate imprecision. Although only studies involving diabetic patients were included, most studies were conducted in tertiary health care settings: eye or diabetes clinics. These settings can afford tabletop or portable fundus cameras. Instead, smartphone ophthalmoscopy is more relevant for screening in primary settings or resource-constrained countries. All studies required mydriasis, despite the availability of nonmydriatic smartphone ophthalmoscopy attachments [17].

Future studies on smartphone ophthalmoscopy could utilize more consistent reference standards, such as the gold standard 7-field ETDRS stereoscopic color photographs, and standardize the DR classification criteria. Such standardization minimizes bias and heterogeneity between studies. In addition, ultrawide-field retinal imaging may detect DR features outside the 7-field ETDRS field of view, which may be of clinical significance [47]. More studies could focus on (1) indirect ophthalmoscopy or ultrawide-field retinal imaging; (2) nonmydriatic techniques; (3) AI; and (4) primary health care settings where the comorbidities and prevalence of DR in this demographic differs.

Smartphone ophthalmoscopy may have an important role in identifying DR in areas with limited access to expensive retinal imaging equipment and trained staff. Our findings show that smartphone ophthalmoscopy performs well in detecting DR. However, the included studies were scarce and heterogeneous and provided imprecise findings. Future studies should use more consistent reference standards and DR classification criteria, evaluate other available forms of smartphone ophthalmoscopy, and recruit participants from primary care settings.