This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Diabetic retinopathy (DR), a common complication of diabetes mellitus, is the leading cause of impaired vision in adults worldwide. Smartphone ophthalmoscopy involves using a smartphone camera for digital retinal imaging. Utilizing smartphones to detect DR is potentially more affordable, accessible, and easier to use than conventional methods.
This study aimed to determine the diagnostic accuracy of various smartphone ophthalmoscopy approaches for detecting DR in diabetic patients.
We performed an electronic search on the Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library for literature published from January 2000 to November 2018. We included studies involving diabetic patients, which compared the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR to an accurate or commonly employed reference standard, such as indirect ophthalmoscopy, slit-lamp biomicroscopy, and tabletop fundus photography. Two reviewers independently screened studies against the inclusion criteria, extracted data, and assessed the quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies–2 tool, with disagreements resolved via consensus. Sensitivity and specificity were pooled using the random effects model. A summary receiver operating characteristic (SROC) curve was constructed. This review is reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies guidelines.
In all, nine studies involving 1430 participants were included. Most studies were of high quality, except one study with limited applicability because of its reference standard. The pooled sensitivity and specificity for detecting any DR was 87% (95% CI 74%-94%) and 94% (95% CI 81%-98%); mild nonproliferative DR (NPDR) was 39% (95% CI 10%-79%) and 95% (95% CI 91%-98%); moderate NPDR was 71% (95% CI 57%-81%) and 95% (95% CI 88%-98%); severe NPDR was 80% (95% CI 49%-94%) and 97% (95% CI 88%-99%); proliferative DR (PDR) was 92% (95% CI 79%-97%) and 99% (95% CI 96%-99%); diabetic macular edema was 79% (95% CI 63%-89%) and 93% (95% CI 82%-97%); and referral-warranted DR was 91% (95% CI 86%-94%) and 89% (95% CI 56%-98%). The area under SROC curve ranged from 0.879 to 0.979. The diagnostic odds ratio ranged from 11.3 to 1225.
We found heterogeneous evidence showing that smartphone ophthalmoscopy performs well in detecting DR. The diagnostic accuracy for PDR was highest. Future studies should standardize reference criteria and classification criteria and evaluate other available forms of smartphone ophthalmoscopy in primary care settings.
Diabetic retinopathy (DR) is the leading cause of impaired vision worldwide [
Diabetic eye disease is treatable. Treatments include vascular endothelial growth factor inhibitors, panretinal or focal photocoagulation, and vitrectomy [
The gold standard diagnostic test for DR is the Early Treatment Diabetic Retinopathy Study (ETDRS) 7-field stereoscopic color fundus photography or fluorescein angiography [
Smartphone ophthalmoscopy, the use of a smartphone’s in-built camera for retinal imaging, could be a valuable method for detecting DR because of its affordability, portability, and ease of use compared with traditional approaches [
This scoping review was reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines [
We performed a librarian-assisted search on the Medical Literature Analysis and Retrieval System Online (MEDLINE) (Ovid), EMBASE (Ovid), and the Cochrane Library for papers published from January 2000 to November 2018. Articles published before 2000 were excluded because before that smartphone technology was limited. We used both medical subject headings (MeSH) and keywords relating to DR (eg, “diabetic retinopathy,” “macular edema,” and “diabetic maculopathy”) and to smartphones (eg, “mobile health,” “mobile phones,” and “applications”) or AI (eg, “artificial intelligence” and “machine learning”;
The inclusion criteria were as follows: (1) studies evaluating the diagnostic test accuracy of smartphone ophthalmoscopy for detecting DR in patients with type 1 or 2 DM; (2) studies utilizing a smartphone’s in-built camera for retinal imaging, including the use of any attachments externally fitted to the smartphone; (3) studies comparing smartphone ophthalmoscopy with any acceptable and commonly employed reference standard, such as fundus photography, indirect ophthalmoscopy, slit-lamp biomicroscopy, or fluorescein angiography; (4) studies employing any kind of health care professional to acquire the smartphone images. Language was not an exclusion criterion.
Examples of eligible smartphone ophthalmoscopy techniques include the following:
Direct ophthalmoscopy: An adaptor is externally attached to a smartphone’s camera. These adaptors usually contain polarizers that reduce artifacts from corneal reflections. The arrangement of polarizers, beam-splitters, and lenses produces an annular illumination pattern.
Indirect ophthalmoscopy: This simpler, monocular design involves a single lens (eg, 20 D condenser) placed between the smartphone camera and eye. It can be mounted on the phone via hardware or manually held in position.
Covidence software (Veritas Health Innovation, Melbourne, Australia) was used to remove duplicated studies [
A data extraction form (
The Quality Assessment of Diagnostic Accuracy Studies tool, QUADAS-2, consisting of descriptions and signaling questions, was used to assess the risk of bias and applicability of all included studies in four domains pertaining to (1) patient selection, (2) index test, (3) reference standard, and (4) flow of participants through the study and timing between the index test and reference standard [
We constructed 2×2 tables based on data from each study. The sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR−), diagnostic odds ratio (DOR), and area under summary receiver operating characteristic (SROC) curve were calculated using a random effects model because of the high expected heterogeneity [
Heterogeneity was evaluated using chi-square (χ²) and I2 values of likelihood ratio tests (LRT) or DOR, with I2<25%, 25–75%, and >75% representing low, moderate, and high degree of inconsistency, respectively. Threshold effect was measured using the Spearman correlation coefficient ρ between logits of sensitivity and specificity, with ρ closer to −1 indicating higher threshold effect and better fit of the SROC curve. If information regarding a condition’s prevalence was available from the literature, we calculated the posttest probability using the Fagan nomogram. A
Our search strategy yielded 1571 unique records. Of those records, the full text for 41 articles was assessed, and nine studies [
All studies reported smartphone fundoscopy techniques involving mydriatic, color, and nonstereoscopic imaging (
A total of five studies [
Flowchart depicting the identification of relevant studies.
Characteristics of included studies.
Study author, year | Country, setting | Sample size (patients/eyes) | Age (years), mean (SD) | Diabetes duration (years) | Diabetic retinopathy severity scale | Reference standard | |||
|
|
|
|
Mean (SD) | Range |
|
|
||
Bhat, 2016 [ |
N/Aa | 80/N/A | N/A | N/A | N/A | ICDRb severity scale; no referral defined as no or mild signs of DRc. | Slit-lamp exam | ||
Kim, 2017 [ |
United States, Retina Clinic | 72/144 | N/A | N/A | N/A | Referable DR defined as moderate NPDRd or worse, or DMEe. | Slit-lamp biomicroscopy | ||
Kim, 2018 [ |
United States, Retina Clinic | 71/142 | 56.7 (16.9) | N/A | N/A | Airlie House ETDRSf criteria; RWDRg defined as moderate NPDR or worse, or DME. | Gold standard dilated eye examination, with optical coherence tomography for DME | ||
Rajalakshmi, 2015 [ |
India, Tertiary care diabetes hospital | 301/602 | 53.5 (9.6) | 12.5 (7.3) | N/A | Modified ETDRS criteria; STDRh defined as PDRi or DME | Mydriatic 7-standard field digital retinal photography | ||
Rajalakshmi, 2018 [ |
India, Tertiary care diabetes hospital | 301/602 | N/A | N/A | N/A | ICDR severity scale; STDR defined as severe NPDR, PDR, or DME; RDRj defined as moderate NPDR or worse, or DME. | Remidio Fundus On Phone images graded by ophthalmologists | ||
Russo, 2015 [ |
Italy, Diabetic center | 120/240 | 58.8 (16.4) | 11.6 (9.7) | N/A | ICDR severity scale; ETDRS criteria for DME; RWDR defined as moderate NPDR or worse, regardless of DME status. | Dilated slit-lamp biomicroscopy by a retinal specialist | ||
Ryan, 2015 [ |
India, Ophthalmology clinic of a tertiary diabetes care center | 300/600 | 48.0 (11.0) | N/A | 0.1-37.2 years | Modified ETDRS criteria; VTDRk defined as severe NPDR or worse, or DME. | Mydriatic 7-field fundus photography by trained optometrists | ||
Sengupta, 2018 [ |
India, Aravind Eye Hospital | 135/233 | 54.1 (8.3) | 10.7 (5.1) | N/A | National Health Service guidelines; VTDR defined as R2-level or worse (severe NPDR, PDR), or DME. | Dilated slit-lamp biomicroscopy (+90 D lens) and indirect ophthalmoscopy by retinal specialists | ||
Toy, 2016 [ |
United States, Health care safety-net ophthalmology clinic | 50/100 | 60.5 (10.6) | 11.9 (8.4) | N/A | ICDR severity scale; RWDR defined as moderate NPDR or worse, or ungradable images. | Slit-lamp exam + dilated ophthalmoscopy by technicians |
aN/A: not available.
bICDR: International Clinical Diabetic Retinopathy.
cDR: diabetic retinopathy.
dNPDR: nonproliferative diabetic retinopathy.
eDME: diabetic macular edema.
fETDRS: Early Treatment Diabetic Retinopathy Study.
gRWDR: referral-warranted diabetic retinopathy.
hSTDR: sight-threatening diabetic retinopathy.
iPDR: proliferative diabetic retinopathy.
jRDR: referable diabetic retinopathy.
kVTDR: vision-threatening diabetic retinopathy.
Description of smartphone ophthalmoscopy imaging techniques.
Study author, year | Attachment used | Imaging technique | Smartphone used | Ungradable |
Bhat, 2016 [ |
Ocular Cellscope | Up to five fields, 50°; AIa |
iPhone 5S | N/Ab |
Kim, 2017 [ |
Cellscope Retina | Both human and AI (EyeApp) graders employed. | N/A | N/A |
Kim, 2018 [ |
Cellscope Retina | 5-field, 50°; fields imaged: central, inferior, superior, nasal, and temporal retina; images were digitally stitched, creating a 100° image; pixels per retinal degree: 52.3; acquired by: medical students or interns. | iPhone 5S | 2 (1.7%) images/eyes |
Rajalakshmi, 2015 [ |
Remidio Fundus on Phone (FOP) | 4-field, 45°; fields imaged: macula, disc and nasal to optic disc, superior-temporal, inferior-temporal retina; autofocus function of smartphone was used. | Android phone | 0 |
Rajalakshmi, 2018 [ |
Remidio Fundus on Phone (FOP) | 4-field, 45°; fields imaged: macula centered, disc centered, superior-temporal, and inferior-temporal retina; AI: EyeArt software used to grade images. | Android phone | 5 (1.7%) patients |
Russo, 2015 [ |
D-Eye (Si14 SpA, Padova, Italy) | 20°; videography and digital images acquired, comprising the posterior pole, macula, optic disc, and peripheral retina; resolution: 3264×2448 pixels; pixels per retinal degree: 150; acquired by: a retinal specialist. | iPhone 5 | 9 (3.8%) eyes |
Ryan, 2015 [ |
20 D lens | Videography and then screenshots to obtain the best images of optic nerve and macula; resolution: 3264×2488 pixels; FilmIc Pro app used to adjust focus and zoom independently; acquired by: a medical student with limited training. | iPhone 5 | 11 (1.8%) photographs |
Sengupta, 2018 [ |
Remidio FOP | 3-field, 45°; fields imaged: posterior pole (macula centered), nasal, and superotemporal field; resolution: 441 pixels per inch; acquired by: ophthalmic photographer without special training. | HTC One (M8) | 1.7-2.1% of images |
Toy, 2016 [ |
Volk Digital ClearField lens mounted on Paxos Scope posterior segment hardware adapter | Variable number of fields, 45°; acquired by: an ophthalmologist. | iPhone 5S | 2 (2%) eyes |
aAI: artificial intelligence.
bN/A: not available.
We carried out the quality assessment of the included studies using the QUADAS-2 criteria (
Quality of included studies assessed via Quality Assessment of Diagnostic Accuracy Studies–2 tool.
In all, six studies (977 participants;
We performed subgroup analysis by removing studies individually and investigating the effect on both I2 and ρ. When one study [
Forest plot of the sensitivity and specificity of smartphone ophthalmoscopy in detecting different grades of diabetic retinopathy. AI: artificial intelligence; FN: false negatives; FP: false positives; NPDR: nonproliferative diabetic retinopathy; PDR: proliferative diabetic retinopathy; RWDR: referral-warranted diabetic retinopathy; STDR: sight-threatening diabetic retinopathy; TN: true negatives; TP: true positives; VTDR: vision-threatening diabetic retinopathy.
Summary receiver operating characteristic curves of smartphone ophthalmoscopy in detecting (A) any diabetic retinopathy; (B) mild nonproliferative diabetic retinopathy; (C) moderate nonproliferative diabetic retinopathy; (D) severe nonproliferative diabetic retinopathy; (E) proliferative diabetic retinopathy; (F) diabetic macular edema; (G) referral-warranted diabetic retinopathy, vision-threatening diabetic retinopathy, or sight-threatening diabetic retinopathy; (H) artificial intelligence to detect referral-warranted diabetic retinopathy. HSROC: hierarchical summary receiver operating characteristic.
Summary of smartphone ophthalmoscopy’s test accuracy in detecting different grades of diabetic retinopathy.
DRa staging | Studies, n | Overall pooled sensitivity, % (95% CI) | Overall pooled specificity, % (95% CI) | Positive likelihood ratio (95% CI) | Negative likelihood ratio (95% CI) | Diagnostic odds ratio (95% CI) | Area under summary receiver operating characteristic curve (95% CI) |
Any DR | 6 | 87 (74-94) | 94 (81-98) | 14 (4.4–44) | 0.14 (0.06-0.29) | 100 (27.4-368) | 0.957 (0.936-0.972) |
Mild NPDRb | 4 | 39 (10-79) | 95 (91-98) | 8.6 (3.6-20) | 0.64 (0.32-1.3) | 13.6 (3.14-58.5) | 0.939 (0.915-0.957) |
Moderate NPDR | 4 | 71 (57-81) | 95 (88-98) | 15 (4.9-43) | 0.31 (0.20-0.49) | 46.9 (10.6-208) | 0.879 (N/A) |
Severe NPDR | 5 | 80 (49-94) | 97 (88-99) | 28 (6.1-133) | 0.21 (0.069-0.65) | 134 (17.5-1040) | 0.965 (0.945-0.978) |
PDRc | 5 | 92 (79-97) | 99 (96-99) | 97 (22-425) | 0.079 (0.027-0.23) | 1225 (117-12,800) | 0.979 (N/A) |
DMEd | 4 | 79 (63-89) | 93 (82-97) | 11 (4.2-30) | 0.22 (0.12-0.42) | 49.8 (13.7-180) | 0.925 (0.898-0.945) |
RWDRe (moderate NPDR or worse) | 4 | 91 (86-94) | 89 (56-98) | 8.1 (1.6-41) | 0.11 (0.072-0.16) | 75.8 (13.9-414) | 0.921 (0.894-0.941) |
RWDR, VTDRf, STDRg | 6 | 87 (77-92) | 96 (71-99) | 24 (2.6-226) | 0.14 (0.087-0.23) | 171 (25.9-1142) | 0.929 (0.903-0.949) |
AIh (RWDR) | 2 | 91 (84-96) | 50 (38-62) | 1.8 (1.4-2.3) | 0.17 (0.088-0.32) | 11.3 (4.92-26.1) | N/Ai |
aDR: diabetic retinopathy.
bNPDR: nonproliferative diabetic retinopathy.
cPDR: proliferative diabetic retinopathy.
dDME: diabetic macular edema.
eRWDR: referral-warranted diabetic retinopathy.
fVTDR: vision-threatening diabetic retinopathy.
gSTDR: sight-threatening diabetic retinopathy.
hAI: artificial intelligence.
iN/A: not available.
In all, four studies (542 participants) presented data on detecting mild NPDR [
A total of four studies (542 participants) presented data on detecting moderate NPDR [
One study [
Overall, five studies (677 participants) presented data on detecting severe NPDR [
Removing one study [
A total of five studies (677 participants) presented data on detecting PDR [
Removing one study [
Although the diagnosis of DME generally requires stereoscopic retinal imaging, these studies used substitute markers, such as the presence of hard exudates or laser photocoagulation scars.
In all, four studies (627 participants) presented data on detecting DME [
In all, four studies (313 participants) presented data on detecting RWDR [
Overall, six studies (914 participants) presented data on detecting RWDR, VTDR, and STDR [
One study excluded from the analysis found the agreement for detecting VTDR to be high, κ=0.76 (95% CI 0.68-0.85) [
In all, two studies (152 participants) presented data on detecting RWDR using AI to grade retinal images acquired via smartphone ophthalmoscopy compared with conventional slit-lamp biomicroscopy [
Another study (301 participants) compared an AI’s grading of smartphone ophthalmoscopy images with the reference standard ophthalmologists’ grading of the same images [
Overall, smartphone ophthalmoscopy performed well in detecting DR. Depending on the severity of DR, smartphone ophthalmoscopy had different accuracy. Progressing from mild NPDR to PDR, we observed an increasing trend in smartphone ophthalmoscopy’s sensitivity, specificity, and DOR. In addition, smartphone ophthalmoscopy had the best performance in detecting PDR, RWDR, VTDR, and STDR; these are important categories to detect as they can significantly affect vision. The lowest sensitivity was observed for detecting mild NPDR, mainly caused by one study enrolling only 7 participants with RWDR. The DOR was lowest for AI’s detection of RWDR. There was also a low percentage of ungradable images across most studies, implying that smartphone ophthalmoscopy is relatively reliable. Common causes of ungradable images included cataracts, poor pupil dilation, vitreous hemorrhages, or poor image focus.
Most studies performed smartphone direct ophthalmoscopy utilizing one of four different attachments. The included studies also assessed two methods of indirect ophthalmoscopy. Smartphone ophthalmoscopy in the included studies surpasses the UK NHS targets requiring DR retinal imaging equipment to have a minimum resolution of 6 megapixels or 30 pixels per retinal degree [
The diagnostic accuracy of AI in grading smartphone ophthalmoscopy images was unexpectedly low. In two studies, the specificity and DOR of AI in detecting RWDR was lower than that of human graders (retinal specialists and ophthalmologists). Nevertheless, one of those studies employed both human and AI to grade identical smartphone-acquired images; the specificity of AI was higher than that of humans. In contrast, a 2015 study demonstrated that AI detects RWDR in smartphone ophthalmoscopy images with 100% sensitivity and 80% specificity (AUC 0.94) [
Smartphone ophthalmoscopy is a safe means of acquiring retinal images [
To our knowledge, this is the first meta-analysis evaluating the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR in diabetic patients. A meta-analysis [
This scoping review aimed to provide a comprehensive analysis of the available literature in this field. Correspondingly, we had broad inclusion criteria encompassing different smartphone ophthalmoscopy techniques, reference standards, DR severity scales, and health care professionals. Smartphone retinal imaging is an emerging technology, and we wanted to capture as much of the available evidence as possible (
Our study employed a comprehensive search strategy and examined studies from different countries involving different types of diabetic patients. At least two reviewers performed quality assessment and data extraction independently following Cochrane methodology. Based on the QUADAS-2 tool, most studies possessed minimal risk of bias and little applicability concerns. In particular, all included studies blinded or masked the graders.
Although the protocol for this scoping review was published in
The large 95% CIs for most SROC curves indicate imprecision. Although only studies involving diabetic patients were included, most studies were conducted in tertiary health care settings: eye or diabetes clinics. These settings can afford tabletop or portable fundus cameras. Instead, smartphone ophthalmoscopy is more relevant for screening in primary settings or resource-constrained countries. All studies required mydriasis, despite the availability of nonmydriatic smartphone ophthalmoscopy attachments [
Future studies on smartphone ophthalmoscopy could utilize more consistent reference standards, such as the gold standard 7-field ETDRS stereoscopic color photographs, and standardize the DR classification criteria. Such standardization minimizes bias and heterogeneity between studies. In addition, ultrawide-field retinal imaging may detect DR features outside the 7-field ETDRS field of view, which may be of clinical significance [
Smartphone ophthalmoscopy may have an important role in identifying DR in areas with limited access to expensive retinal imaging equipment and trained staff. Our findings show that smartphone ophthalmoscopy performs well in detecting DR. However, the included studies were scarce and heterogeneous and provided imprecise findings. Future studies should use more consistent reference standards and DR classification criteria, evaluate other available forms of smartphone ophthalmoscopy, and recruit participants from primary care settings.
Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library search strategy.
Data extraction sheet.
Additional details of included studies.
Details of quality assessment of included studies using Quality Assessment of Diagnostic Accuracy Studies-2.
Additional study details received from study investigators.
artificial intelligence
area under curve
diabetes mellitus
diabetic macular edema
diagnostic odds ratio
diabetic retinopathy
Early Treatment Diabetic Retinopathy Study
false negatives
false positives
negative likelihood ratio
positive likelihood ratio
likelihood ratio test
National Health Service
nonproliferative diabetic retinopathy
proliferative diabetic retinopathy
Quality Assessment of Diagnostic Accuracy Studies
referral-warranted diabetic retinopathy
summary receiver operating characteristic
sight-threatening diabetic retinopathy
true negatives
true positives
vision-threatening diabetic retinopathy
The authors thank Ms Soong Ai Jia for her contribution to the data extraction stage of this review. The authors gratefully acknowledge funding support from Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.
LTC conceived the idea for this study. CHT and BK screened the articles, extracted the data, and performed the analysis. CHT and LTC wrote the manuscript. BK, HS, CT, and LTC revised the manuscript critically.
None declared.