This study was approved by the study hospitals’ institutional ethics committees (2022-KY-143) and adhered to the tenets of the Declaration of Helsinki. All parents or guardians of the recruited infants provided written informed consent prior to participation. Data were anonymized and deidentified before analysis.
We collected data retrospectively on infants who received anti-VEGF monotherapy for ROP requiring treatment between April 2016 and November 2022 at Hospital 1 (Zhujiang Hospital, Southern Medical University), Hospital 2 (The Third Affiliated Hospital, Guangzhou Medical University), and Hospital 3 (Guangzhou Children’s Hospital and Guangzhou Women and Children’s Medical Center), all in Guangzhou, China. Infants with incomplete data or any other ocular diseases besides ROP were excluded. Additionally, infants who underwent anti-VEGF therapy as adjunctive treatment before planned vitrectomy or received follow-up examinations for less than 12 months were excluded.
During each ROP screening examination, retinal photographs were captured using the RetCam Ⅲ digital fundus camera (Natus). The diagnosis of ROP and the indication for its treatment were based on the International Classification of ROP Revisited and the Early Treatment for ROP study, respectively. Treatment-requiring ROP included threshold disease, stage 4 or 5 ROP, and type 1 or aggressive ROP. Ocular examinations were conducted before and on days 1, 7, 14, and 28 after anti-VEGF therapy, and either biweekly or monthly depending on the ocular findings and systemic conditions. A reactivation of ROP was defined as the recurrence of acute phase features including a range of signs from a new demarcation line to reactivated stage 3 with disease, vascular dilation, tortuosity, or new/recurrent neovascularization that required further treatment [
All parents or guardians of the infants were fully informed of the efficacy and possible complications prior to IVI of conbercept, and they provided written informed consent. The anti-VEGF treatment was performed as monotherapy for treatment-naive patients. Anti-VEGF agents were injected intravitreally at 1.5 mm posterior to the limbus with a 30-gauge needle under topical aesthesia. Topical tobramycin dexamethasone was administered for 3 days after the injection. All operations were done by a trained pediatric ophthalmologist (author SF).
Based on previous studies and clinical experience, the potential clinical risk factors of ROP reactivation extracted from electronic medical records included maternal factors, neonatal factors, ocular conditions, laboratory factors, and neonatal interventions. Specifically, maternal factors of interest included maternal age, gestational hypertension, gestational diabetes mellitus, premature rupture of membranes, cesarean delivery, and in vitro fertilization and embryo transfer. Neonatal factors included gestational age, birth weight, PMA at initial ROP treatment, fetal distress, sex, small for gestational age, Apgar scores (1 and 5 min), multiple births, asphyxia, sepsis, respiratory distress syndrome, bronchopulmonary dysplasia, pneumonia, intraventricular hemorrhage, necrotizing enterocolitis, hypoxic-ischemic encephalopathy, atrial septal defect, patent foramen ovale, patent ductus arteriosus, and hyperbilirubinemia. Ocular conditions included zone I ROP, preretinal hemorrhage, and aggressive ROP. The hemoglobin concentration before treatment was included among the laboratory factors. Finally, neonatal interventions included mechanical ventilation and oxygen therapy (before treatment) [
In the data processing stage, any missing data were handled according to the type of variable. For continuous data, we imputed missing values by calculating the mean within each category based on the label (0 or 1). For discrete data, we used the mode within each category to fill in missing values. This approach ensured that the values imputed were representative of their respective categories, thus minimizing bias in the analysis.
All retinal photographs were captured using the commercial RetCam camera. Retinal photographs of poor photographic quality were excluded by 2 experienced ophthalmologists (Authors SF and RW). Since the prediction for ROP reactivation would be performed independently on each infant rather than each retinal photograph, retinal photographs of both eyes from the same infant were labeled as a single case. All cases were labeled independently by 2 clinical ophthalmologists (SF and RW). Based on the ocular findings after anti-VEGF therapy, each case was annotated reactivation or nonreactivation. If ROP reactivation occurred in one of the eyes, the case would be labeled as reactivation.
The κ was 0.81 for annotation of ROP reactivation, indicating good agreement between the 2 ophthalmologists in labeling. Moreover, the labels were further confirmed by a senior retinal specialist (author XL) to generate a final annotation. These annotations were used as ground-truth labels in the development and validation of the prediction models for ROP reactivation.
An illustration of the conventional machine learning prediction model is presented in
During the data preprocessing stage, we employed 2 filling strategies. For discrete data, we used mean filling followed by rounding; for continuous data, we used mean filling. Subsequently, the continuous data underwent standardization, transforming the data into a standard normal distribution with a mean of 0 (SD 1), ensuring uniform scales across different features. This helped mitigate potential model biases arising from scale differences, enhancing overall model robustness and performance.
In this study, we employed a comprehensive approach to assess the feature importance in predictive models, utilizing 5 different algorithms: random forest (RF), Adaptive Boosting (AdaBoost), Extreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost), and logistic regression (LR). Table S1 in
We conducted an in-depth investigation into the impact of varying the number of selected features (N) on the predictive model performance, employing 5 distinct algorithms. We systematically varied the number of selected features, choosing from the set (5, 10, 15, 20, 25, and 30), and assessed the resulting model performance by calculating the area under the curve (AUC) for each model across different feature subsets.
We conducted a comprehensive evaluation of the 5 different predictive models, with a specific focus on their receiver operating characteristic (ROC) curves. The models were trained and evaluated using all features on a standardized data set. The emphasis of this research lies in presenting the performance of different models through ROC curves, offering crucial insights into their predictive capabilities. Furthermore, we conducted 10 iterations of training for each of the 5 distinct predictive models and computed their average AUC values on the corresponding validation sets, along with their respective SDs.
Illustration of the proposed algorithmic pipeline. CatBoost: Categorical Boosting.
We designed a series of image preprocessing methods beginning with feature enhancement and noise suppression using median filtering. This technique was chosen for its effectiveness in removing isolated noise pixels while maintaining the clarity of important features such as blood vessel edges. Subsequently, the images were converted to grayscale for processing to reduce computational complexity.
For the training set, we employed a series of preprocessing steps. These included resizing the images to 224 × 224 pixels, introducing RandAugment for random data augmentation, and applying both random horizontal and vertical flips. These measures aimed to increase the diversity of the data, enhancing the model's robustness. For the internal validation and test sets, we consistently resized the images to 224 × 224 pixels and transformed them into tensors. This ensured uniformity in the model's input during the validation process. This diversified preprocessing approach helped the model adapt to the distinct characteristics of each data set, ultimately improving its generalization performance.
We employed a pretrained ResNet-50 model and iteratively trained it on the image training set, incorporating image preprocessing and data augmentation techniques to enhance the model's generalization performance. We adopted the cross-entropy loss function, whereby the model learned to adjust its parameters by minimizing the cross-entropy loss, enabling it to more accurately predict the samples’ categories. During the training process, we utilized the Adam optimizer and a cosine annealing learning rate scheduler, conducting a total of 100 epochs. At the end of each epoch, we evaluated the model's performance on both an internal validation set and an external test set. Throughout the training, we also applied the Exponential Moving Average (EMA) technique to stabilize the model parameters and improve the overall model generalization.
We constructed a data set in which each case consisted of multiple medical images and randomly selected 5 images per patient for the experiment. When performing case-level predictions, we treated the model's output as the probability score for the entire case. By averaging the probability scores from multiple images within each case, we computed the final case-level prediction results.
To enhance the visualization and interpretability of the ROP reactivation prediction, we applied the Gradient-Weighted Class Activation Mapping (Grad-CAM) technique to generate heat maps of key regions. This technique highlights areas crucial to ROP reactivation prediction by analyzing feature weights in specific network layers, such as layer 4 of ResNet50. This approach allows us to intuitively identify retinal features that significantly contribute to the prediction outcome.
An illustration of the prediction model combining the conventional machine learning model and the deep learning model is presented in
First, we modeled the basic clinical features of patients using a risk factor model, considering traditional risk factors such as maternal age and infant weight. Subsequently, we employed a deep learning model (ResNet-50) to process retinal images of newborns, capturing more comprehensive information. The deep learning model, by learning features from images, can capture complex patterns that traditional models find challenging.
To fully leverage the strengths of both models, we treated the 5 probability scores generated by the deep learning model for each image as equal in weight and assigned the output of the risk factor model a weight equivalent to that of 4 images, thereby producing a more accurate and robust prediction result. We obtained the deep learning probability scores for each sample through independent training on the training set, simultaneously predicting risk factor probabilities using the risk factor model. Finally, we linearly combined the probability scores from both models to derive the ultimate fusion result.
Statistical analyses were performed using R software (version 3.0.2; R Core Team). Categorical variables were expressed as numbers and percentages and analyzed using chi-square tests. If any cell number was less than 5, the Fisher exact test was applied. Continuous variables conforming to normal distribution were expressed as mean and SD and compared using the independent 2-sample