This study was conducted in Shanghai, China, in 2022. Since 2010, Shanghai has conducted an active community-based fundus disease telemedicine screening program. After 2018, an AI model was adopted (Figure 1). At the end of 2021, the fully automatic fundus camera was adopted, and the facilitated self-service fundus disease screening pattern was established (Figure 1). Within this new pattern, residents could perform fundus photography by themselves without professionals’ assistance (Multimedia Appendix 1). The fundus images were sent to the cloud server center of the AI model, and the screening results were fed back immediately.

We conducted a cross-sectional study at 2 adjacent screening sites. These 2 sites were expected to be very similar in terms of their socioeconomic and educational aspects since they were located next to each other. One site provided facilitated self-service fundus disease screening, and the residents who participated therein comprised the exposure group; the other site provided screening with a traditional fundus camera operated by an optometrist, and the residents who participated therein comprised the control group. All the adult residents could participant in our screening program, but their data were used for analysis only if they signed the informed consent form. Residents could opt out of the study at any time during the screening.

In the exposure group, the residents were assessed using an updated version of the nonmydriatic fundus camera Kestrel 3100m (Shanghai Top View Industrial Co Ltd) with a self-service module. In the process of fundus photography, the residents pressed the “Start” button by themselves. All checking steps (including focusing, shooting, and image quality review) were undertaken automatically by the fundus camera (Figure 2). Screening data were transmitted to the AI algorithm on a cloud-based server center through the telemedicine platform, and the screening results were fed back immediately. Residents were fully informed that the assessment was fully automated and not performed by the optometrist.

In the control group, the residents were assessed using the basic version of the same nonmydriatic fundus camera. The optical components were identical to those in the exposure group but without the self-service module. In the process of fundus photography, all steps were carried out by the optometrist (including focusing, shooting, and image quality review). Screening data were transmitted to the AI algorithm on a cloud-based server center through the telemedicine platform, and the screening results were fed back immediately. Residents were also fully informed.

The primary outcome was the screening service’s quality. Based on the World Health Organization’s recommendations for the evaluation of AI-based medical devices [14] and the European Union’s Assessment List for Trustworthy Artificial Intelligence [15], 5 dimensions were selected to reflect the service quality of eye disease screening: effectiveness, physiological discomfort, safety, convenience, and trustworthiness.

Furthermore, effectiveness was based on 2 indicators: image quality and screening efficiency. A staff member recorded the time required for each resident to take fundus photographs (excluding the time taken for diagnosis) at the screening site. Then, a professional ophthalmologist evaluated the quality of each fundus photograph after the on-site experiment. The ophthalmologist was blinded to the grouping of participants. Image quality was assessed on the basis of the image quality pass rate, expressed as the number of eyes with high-quality fundus images per 100 eyes. Screening efficiency was assessed on the basis of the average screening time, expressed as the mean of the time required for each resident to take fundus photographs.

To assess physiological discomfort, safety, convenience, and trustworthiness of screening services, residents were asked to finish a questionnaire just after they received the screening results. A 5-point Likert scale was adopted for each dimension, from the best to the worst, except for the physiological discomfort (Multimedia Appendix 2). A no physiological discomfort rate was expressed as the number of residents who chose the “There is no physiological discomfort during the screening” per 100 individuals in each group. Safety rate is expressed as the number of residents who chose “The screening is very safe” or “The screening is safe” per 100 individuals in each group. Convenience rate is expressed as the number of residents who chose “The screening is very convenient” or “The screening is convenient” per 100 individuals in each group. The trustworthiness rate is expressed as the number of residents who chose “The screening result is very trustworthy” or “The screening result is trustworthy” per 100 individuals in each group.

The secondary outcome was the preference rate, expressed as the number of residents who were willing to use the same technology for their next screening per 100 individuals. In detail, in the exposure group, the preference rate was expressed as the number of the residents who preferred facilitated self-service eye screening per 100 individuals, while in the control group, it was expressed as the number of residents who preferred traditional manual screening per 100 individuals.

To understand the residents’ preference, a video displaying the processes of both facilitated self-service eye screening and traditional manual screening was shown to the residents. Then, the following question was asked: “At your next eye disease screening, you can choose either facilitated self-service eye screening or traditional manual screening. Which one do you prefer?” A total of 4 alternatives were set: “Prefer traditional manual screening,” “Prefer facilitated self-service eye screening,” “Both are acceptable,” and “Neither is acceptable (Refusal of screening).” Each resident could choose only 1 option, which best reflected their preference.

The rule of events per variable was used for sample size estimation. In this study, 2 logit models were established for the 2 groups separately, each containing 8 independent variables. We set 10 events per variable in general. According to a previous study [16], when the decision-making process had high uncertainty, the proportion of individuals who preferred the algorithms was about 50%. This led us to arrive at a sample size of 160 (8 variables multiplied by 10 events each, with 50% of individuals potentially preferring facilitated screening [ie, 50% of 8×10]) for each group.

Every dimension of the screening service quality and the preference rate were calculated separately. Chi-square and t tests were used to test whether the service quality or the residents’ preferences differed between the 2 groups. A total of 7 hypotheses were tested, as shown in Textbox 1.

If any of the hypotheses among hypotheses 1-6 (Textbox 1) were significant, it indicated that the service quality was different between facilitated self-service eye screening and traditional manual screening. If hypothesis 7 was significant, it meant that the residents’ preference for facilitated self-service eye screening was different from that for traditional manual screening.

Additionally, subgroup analyses in the exposure and control groups were conducted to explore the relationships between the screening service quality and the residents’ preferences, using generalized logit models. The option “Prefer facilitated self-service eye screening” was used as the reference level for the dependent variable in the models. The independent variables included age, sex, image quality, screening efficiency, physiological discomfort, safety, convenience, and trustworthiness. All statistics were performed using SAS (version 9.4; SAS Institute).

The study adhered to the ethical principles of the Declaration of Helsinki and was approved by the Shanghai General Hospital Ethics Committee (2022SQ272). All participants provided written informed consent before participating in this study. The study data were anonymous, and no identification of individual participants in any images of the manuscript or supplementary material is possible.

A total of 358 residents enrolled; among them, 176 (49.16%) were in the exposure group and the remaining 182 (50.84%) were in the control group. Residents’ basic characteristics were balanced between the 2 groups. The mean age was 65.05 (SD 12.28) years for the exposure group and 63.96 (SD 13.06) years for the control group; however, this difference was nonsignificant (P=.42). The proportion of women was 67.05% (n=118) for the exposure group and 62.09% (n=113) for the control group; this difference was also nonsignificant between the 2 groups (P=.33).

In the exposure group, high-quality fundus images were obtained for 268 out of 352 eyes (image quality pass rate=76.14%; Figure 3). The average screening time was 81.03 (SD 36.98) seconds (Figure 3). In the control group, high-quality fundus images were obtained for 274 out of 364 eyes (image quality pass rate=75.27%; Figure 3). The average screening time was 78.22 (SD 54.01) seconds (Figure 3). There was no significant difference in the image quality pass rate (χ²₁=0.07, P=.79) and average screening time (t_321.01=–0.58 [Welch–Satterthwaite–adjusted df], P=.56) between the 2 groups (Figure 3).

For the other dimensions, detailed information is shown in Figure 3. There were no significant differences between any of these rates between the 2 groups (no physiological discomfort rate: χ²₁=0.01, P=.92; safety rate: χ²₁=0.08, P=.78; convenience rate: χ²₁=0.004, P=.95; trustworthiness rate: χ²₁=1.63, P=.20).

In the exposure group, 120 (68.18%) residents preferred traditional manual screening, 19 (10.80%) preferred facilitated self-service eye screening, 19 (10.80%) preferred both, and the remaining 18 (10.23%) preferred neither. In the control group, 123 (67.58%) residents preferred traditional manual screening, 14 (7.69%) preferred facilitated self-service eye screening, 20 (10.99%) preferred both, and the remaining 25 (13.74%) preferred neither.

The proportion of residents who chose the category “Prefer facilitated self-service eye screening” in the exposure group was significantly lower than that of residents who chose the category “Prefer traditional manual screening” in the control group (χ²₁=120.57, P<.001; Figure 3).

In the exposure group, 4 generalized logit models were generated (Table 1). Regarding the effectiveness of facilitated self-service eye screening, neither the image quality nor the screening time had an impact on the residents’ preferences. Regarding the other dimensions for facilitated self-service eye screening service quality, models 3 and 4 demonstrated that distrust in the results of facilitated self-service eye screening might decrease the probability of preferring this screening service and increase the probability of preferring neither of the 2 screening services.

In the control group, another 4 generalized logit models were generated (Table 2). Men were more likely to choose a preference both screening services. The probability of preferring manual screening might increase with age, as long as the probability of preferring facilitated self-service eye screening decreased. Regarding the effectiveness of traditional manual screening, neither the image quality pass rate nor the screening time had an impact on the residents’ preferences. For the other dimensions of the quality of traditional manual screening, models 7 and 8 showed that if the residents feel unsafe about traditional manual screening, their preference for traditional manual screening might decrease, and they might turn to facilitated self-service eye screening.