In this study, we introduced Digital FaceDefender, a method designed to enhance privacy protection while supporting auxiliary diagnosis of ocular diseases in eye hospitals. Figure 1 shows the detailed workflow, with the region of interest (ROI) clearly marked.

The research protocol was reviewed and approved by the Institutional Review Board/Ethics Committee of the Fourth Affiliated Hospital of Harbin Medical University (approval 2023-Ethics Review-54; see Multimedia Appendix 2). All participants provided informed consent in accordance with the principles outlined in the Declaration of Helsinki.

To protect privacy, all face images presented in the paper were generated using our image fusion method or masked rather than displaying original face images from our private database.

In this retrospective study, we collected face images from 3 sources: 2 publicly available datasets (Flickr-Faces-HQ [FFHQ] and CASIA-FaceV5) and 1 private dataset from the Fourth Affiliated Hospital of Harbin Medical University. The images from the latter dataset were randomly selected.

Based on their extensive clinical diagnostic experience, two board-certified ophthalmologists (one Chief Physician and one Associate Chief Physician), both licensed by the Ministry of Health of the People’s Republic of China, independently evaluated a random sample of facial images from 1136 healthy individuals in our private database. These ophthalmologists, affiliated with the Joint Shantou International Eye Center and the Department of Ophthalmology at the Fourth Affiliated Hospital of Harbin Medical University, assessed the images using a standardized diagnostic protocol. Only individuals exhibiting clear ocular abnormalities, including esotropia, exotropia, vertical strabismus, and ptosis, were further assessed and included in this study. To ensure diagnostic consistency, the evaluations were performed independently, and inter-rater agreement was quantified using the Cohen Kappa coefficient.

Several preprocessing techniques were used to ensure the consistency and quality of patient images. Initially, facial alignment was performed by rotating the original images based on the line connecting the irises, ensuring the face was centered and horizontally oriented. All images were resized to 1024×1024 RGB (red, green, and blue) color space, the standard required for generating digitally synthesized avatars. If the original image size was smaller than this resolution, a white background was added; if larger, the image was cropped to maintain central facial positioning. Given that patient images are captured under variable conditions, including differences in lighting and background noise, a standard RGB color correction method [17,18] was used to adjust brightness, ensuring consistency with the digitally synthesized avatars’ faces.

The FFHQ dataset, consisting of 70,000 high-quality PNG images with a resolution of 1024×1024 pixels, was used to the style-based generative adversarial network architecture (StyleGAN2) [19] model, which is known for generating realistic digitally synthesized avatars. However, this dataset is biased towards Caucasian faces (69% Caucasian, 4% Black, and 27% other) raising concerns regarding generalizability to more diverse populations [20,21], raising concerns about generalizability to diverse populations.

To address this issue, we used the generator_yellow-stylegan2-config-f.pkl file, which was trained on 13,016 Asian facial images from the SeePrettyFace website [22]. We generated male and female digitally synthesized avatars with varying age progressions, ranging from 5 to 85 years old in increments of 10 years, to simulate realistic age-related facial changes.

Accurate generation of lesion positions and morphologies is critical for medical image generation [23]. While generative adversarial networks (GANs) excel at capturing global features, they often struggle with detailed pathological structures, as suggested by Kazeminia et al [24], and Han et al [25]. To mitigate this, we used Google’s MediaPipe library [26] for facial landmark detection. The Face Mesh detector identifies 3D coordinates for 468 facial landmarks (see Multimedia Appendix 3).

Affine transformations preserve collinearity and parallelism [26], making them suitable for adjusting discrepancies in the eye regions between patient and avatar images. We applied the Open Computer Vision Library (OpenCV) warpAffine() function [27] to adjust the eye regions while preserving the original proportions and positions. These transformed images, referred to as preliminary fusion images (see Figure 1), were then used for the fusion process.

During image fusion, disparities in skin tone, brightness, and unnatural fusion boundaries can arise, potentially affecting diagnostic accuracy. To rectify this, we applied RGB color correction [17,18] to harmonize patient images with avatar facial features. In addition, a Gaussian blur [28-30] was applied to the fused image edges to smooth boundary transitions [31] and create a more natural appearance. Figure 2 shows the detailed process by which the final high-quality fusion images (referred to as FDface) are generated through the fusion of raw RGB facial images and digitally synthesized avatar facial images. The workflow begins with the extraction of landmarks from both the raw images and the digitally synthesized avatars, followed by the image fusion step. This is followed by color correction and background adjustment to ensure consistency and improve visual appeal. Finally, Gaussian filtering is applied to smooth the image, further enhancing its quality and resulting in the final high-quality fused facial images. Furthermore, we generated both DMface and FDface images by applying the DM technique and Digital FaceDefender to the same original images for comparative analysis.

To assess the effectiveness of Digital FaceDefender in reducing reidentification risks while preserving essential facial features necessary for accurate diagnoses, we conducted a comparative analysis against DM using the Detailed Expression Capture and Animation (DECA) [8] framework. Specifically, we used the FLAME model to reconstruct facial images, generating DMface and FDface representations from the CASIA-FaceV5 dataset, which comprises 2500 facial images from 500 individuals, each captured in five different poses. These reconstructed images were labeled as FLAME_DMface and FLAME_FDface, respectively.

To evaluate the reduction in reidentification risk associated with Digital FaceDefender, we used ArcFace (IR-SE50 model) to compute facial similarity scores and Rank-1 accuracy. Specifically, we separately quantified the mean similarity scores between FDface and FLAME_FDface images and their corresponding original facial images. These similarity scores indicate the extent to which the generated images resemble the original faces. In addition, we used Rank-1 accuracy to assess whether FDface and FLAME_FDface images could be reidentified as their original counterparts using ArcFace.

To determine the sample size required for constructing the diagnostic benchmark, we first conducted a pilot study with npilot=50 to estimate the SD (σpilot) of the difference in pixel distances along the x-axis between the left and right eyes. The margin of error was then calculated by using the formula (E=Zα2×σpilotnpilot=1.96×3.3150=0.92 , where Zα2=1.96 correspond to a 95% confidence level). Based on these estimates, the final required sample size (n) was determined using the standard sample size formula:

Finally, a total of 1136 healthy individuals were included in our study, exceeding the estimated sample size (n=60) and ensuring sufficient statistical power for establishing the diagnostic benchmark.

Pixel distances (|A|, |B|, |CD|, |EF|, |GH|, and |IJ|) along the x- and y-axes for both eyes were calculated between the iris center (o and o’) and reference points (a, b, c, d, e, f, g, h, i, j) in 1136 healthy individuals (see Figures 3II,3). The difference in pixel distances along the x-axis between the left and right eyes was denoted as |A-B|. Outlier detection was performed using the IQR method, excluding data points outside 1.5 times the IQR from the lower and upper quartiles. After outlier removal, 95% CIs for the pixel distances within the eye region were calculated. For variables that followed an approximately normal distribution (assessed by visual inspection and normality tests), we used the normal distribution method to calculate CIs. For variables that deviated from normality, we applied the bootstrap method to obtain more robust CI estimates. These diagnostic benchmarks were validated on an independent dataset of 114 individuals not included in the original sample. Inter-rater consistency between the diagnostic benchmarks and ophthalmologists’ assessments was evaluated using Cohen Kappa. The significance level was set to .05. Statistical analysis was performed using RStudio (version 1.1.463; Posit), which is developed and maintained by Posit.

The established diagnostic benchmarks were then used to identify ocular abnormalities, such as esotropia and exotropia (see Multimedia Appendix 4).

We developed the Digital FaceDefender platform to support ophthalmologists and researchers in their work, offering a user-friendly interface for privacy protection and auxiliary diagnostic assistance.

Figure S1 in Multimedia Appendix 5 illustrates certain challenges due to low resolution (640×480), nonideal shooting angles, and intricate backgrounds, such as a blue curtain. These factors could hinder accurate representation of critical eye regions. As a representative example, we enhanced Figure S1A to a high-resolution version (2436×1944) shown in Figure S1B, both in Multimedia Appendix 5. Additional corrections included aligning the pupils horizontally in Figure S1C in Multimedia Appendix 5 and refining the cropping to emphasize the central facial region more clearly in Figure S1D in Multimedia Appendix 5.

As depicted in Multimedia Appendix 6, we generated 2 sets of digitally synthesized avatars representing Asian males and females aged from 5 to 85 years at 10-year intervals. These avatars are used as templates in subsequent image fusion processes to ensure the preservation of critical facial features essential for medical diagnoses. Furthermore, a set of digitally synthesized avatars representing Caucasian individuals was also created.

Using Google’s MediaPipe library, we focus on 52 key landmarks within the eye and periocular regions, including the eyelids, eye sockets, irises, and brow arches, for both patients and avatars. These landmarks are used to define closed ROIs for subsequent image fusion. OpenCV is then applied to adjust the eye regions, resulting in transformed images, referred to as preliminary fusion images (see Figure 1), which are subsequently used in the fusion process.

Figure 4 presents the final fused images generated by both the DM method and our proposed Digital FaceDefender approach. The DMface images produced using the DM method are shown in Figure 4C and D, while the FDface images created by Digital FaceDefender are depicted in Figure 4E and F. The difference in fused images generated by DM and Digital FaceDefender highlights that Digital FaceDefender not only meets the privacy protection requirements but also preserves essential features related to ocular diseases. The resulting natural-looking fused images enhance clinician-patient empathy, which is crucial for accurate diagnosis and personalized treatment. Furthermore, the final fused images generated using the various ethnic digitally synthesized avatars, as presented in Multimedia Appendix 7, demonstrate the generalizability of the proposed Digital FaceDefender methodology.

Figure 4 illustrates the difference in reidentification risk between the DM method and the Digital FaceDefender approach. When applying the FLAME reconstruction technique to both DMface and FDface images, the FLAME_DMface images (Figure 4G and H) exhibit a higher reidentification risk compared to FLAME_FDface images (Figure 4I and J). This increased risk stems from the preservation of key facial contours, including the face shape, nose, eyes, and mouth, which enables effective reconstruction through FLAME in the DM method.

As shown in Table 1 and Multimedia Appendices 8,9 and 9, the mean similarity score between FDface and raw facial images is 0.31. While this value suggests that FDface retains some identifiable facial features, potentially beneficial for auxiliary diagnosis, it remains below the reidentification threshold established by leading facial recognition systems. For example, Microsoft’s Face API reports that similarity scores above 0.5 indicate a potential match [32]. The significantly lower mean similarity score of 0.09 for FLAME_FDface versus raw facial images suggests that reconstructing the original face from FDface images is highly challenging, confirming that Digital FaceDefender significantly reduces reidentification risk while enhancing privacy protection.

Further validation comes from Rank-1 accuracy analysis across different facial poses (see Tables 2 and 3). Notably, for Pose #2 and Pose #3 (see Table 2) and Poses #2-#5 (see Table 3), the Rank-1 accuracy is 0, indicating that no successful reidentifications occurred in these conditions. Even for Pose #4 (0.02) and Pose #5 (0.04), the Rank-1 accuracy remains near zero, confirming that Digital FaceDefender effectively mitigates reidentification risks across various facial angles.

These results underscore the robustness of our proposed Digital FaceDefender in balancing privacy protection with retention of diagnostic features.

Using the developed diagnostic benchmarks, we evaluated their agreement with ophthalmologists’ assessments for ocular disease detection (see Figure 5). As shown in Table 4, Cohen Kappa analysis demonstrated a fair agreement for the left eye (κ=0.37) and a moderate agreement for the right eye (κ=0.445). Both agreements were statistically significant (P<.001), indicating the reliability of the benchmarks. However, factors such as head posture and image quality may influence diagnostic accuracy.

The Digital FaceDefender platform is available as a website [33]. Multimedia Appendix 10, along with the accompanying Multimedia Appendix 11, demonstrates the effectiveness of the Digital FaceDefender Platform in protecting patients’ facial privacy while simultaneously supporting the diagnosis of ocular diseases. In addition, the Digital FaceDefender platform utilizes the developed diagnostic benchmarks to identify raw images with suboptimal camera angles, providing text prompts to flag them as poor-quality images.