The objective of this multicenter retrospective diagnostic study was to develop and validate a 2 stage DL facial recognition classification model, GVisageNet, using frontal facial photographs to distinguish patients with midline brain tumors and normal controls (NCs), and further distinguish iGCTs from other patients with midline brain tumors, thereby facilitating early detection. Data collection occurred across 4 Chinese medical institutes from January 2010 to May 2023. The study design and participant flow are illustrated in Figure 1.

Inclusion criteria were (1) patients diagnosed pathologically with midline brain tumors (specify tumors originating in the sellar region, pineal region, or the fourth ventricle); (2) availability of frontal facial photographs taken within 2 weeks before the date of diagnosis; (3) gender- and age-matched NCs from the “Wild Young Labeled Faces” dataset with age and gender matching from the GitHub repository managed by the Institute of Systems and Robotics at the University of Coimbra. Exclusion criteria were (1) patients lacking essential clinical information, including but not limited to demographics (eg, age and gender), symptoms, and hormonal levels; (2) patients with facial photographs that are significantly blurred, to the extent that key facial landmarks or features are indistinguishable; (3) patients who have undergone treatments for other conditions that could significantly affect facial features or hormonal levels; and (4) patients from whom obtaining informed consent is not possible. Finally, patients were categorized into the midline brain tumors group and NCs, and further divided into iGCTs and the other midline brain tumor groups.

This investigation was systematically conducted in 3 phases, using a multicenter approach for patient recruitment and data collection. During the initial phase, NCs and patients diagnosed with midline brain tumors (including iGCTs and other midline brain tumors) were recruited from the wild young labeled faces dataset, as well as from Tiantan, Puren Hospital, Capital Medical University. This can be deleted from the text. In this phase, our goal was to construct a binary tandem DL model (GVisageNet). We randomly divided the NCs and patients with midline brain tumors (n=1016) into 2 groups, 75% for training the GvisageNet and 25% for testing its performance (762:254). Furthermore, an independent external dataset was compiled from 4 distinct medical institutions (Tiantan, Puren, Shijitan, and Sanbo Hospital, Capital Medical University) between January 2020 and May 2023, aiming to establish an independent validation dataset (n=436) for validating the DL model. Subsequently, the patients with midline brain tumors (n=716) from the first phase were randomly divided into a training dataset and a testing dataset in a 75:25 ratio to differentiate iGCTs from other midline brain tumor patients (574:142). Finally, the facial photographs of patients with midline brain tumors (n=236) from the independent external validation dataset were used as a separate validation dataset in the second phase.

Since both iGCTs and other midline brain tumors affect the brain's midline functional areas in clinical diagnosis, leading to similar facial characteristics caused by impacts on the patient’s endocrine function, differential diagnosis becomes even more challenging. However, incorrect predictions could result in misdiagnosis, misinterpretation, and inefficient use of resources. To further enhance predictive accuracy, the subsequent phase of the study aimed to develop a regression model using clinical variables marked by statistical disparities between the 2 groups (iGCTs vs other midline brain tumors). This included demographic characteristics, clinical manifestations, and hormonal profiles, where significant differences were observed. This model was then integrated with the outputs generated by GVisageNet (from the second stage), enabling the construction of an integrated hybrid model. The primary goal of this integration was to clarify the contribution of clinical data in improving the model’s predictive performance.

In the final phase of the study, the correlation analysis between the outputs of GVisageNet and 15 endocrine indicators in the independent validation dataset was conducted to explore the potential mechanism of GVisageNet. Finally, stratified studies were carried out in the independent validation dataset based on the degree of hypothalamic-pituitary-target (HPT) axis damage (complete damage, incomplete damage, and no damage), further verifying that endocrine changes are the potential mechanism behind the effectiveness of GVisageNet.

Frontal views of each patient were routinely captured at the initial hospital visit using a standardized protocol. This involved a digital camera with a resolution exceeding 10 million pixels (the criteria and details of photograph taking are presented in Method S1 in Multimedia Appendix 1).

In studying the correlation between facial recognition features and endocrine functions, the selection of specific clinical indicators is designed to comprehensively assess patients’ endocrine conditions and their potential impacts on facial characteristics. This includes baseline data such as gender, age, BMI, initial symptoms, disease duration, and the primary site of the tumor, which provide an initial understanding of the patient's overall health and endocrine function. Key endocrine indicators include the adrenal axis (Adrenocorticotropic hormone), thyroid axis (thyroid stimulating hormone [TSH], triiodothyronine, free triiodothyronine [FT3], thyroxine, and free thyroxine [FT4]), growth hormone-insulin-like growth factor-I axis (growth hormone [GH], insulin-like growth factor 1, insulin-like growth factor binding protein 3 [IGFBP-3]), and gonadal axis (follicle-stimulating hormone [FSH], luteinizing hormone [LH], testosterone, estradiol, progesterone, and prolactin). Variations in these hormone levels can lead to significant changes in facial features, such as puffiness, growth retardation, or precocious puberty. In addition, routine monitoring of 24-hour urine volume, specific gravity, and osmolality indirectly reflects the patient's endocrine health status, thereby supporting the potential application of facial recognition technology in clinical diagnostics. This comprehensive data collection is crucial for exploring the link between facial characteristics and endocrine functions, holding significant academic value.

The criterion for correct GVisageNet prediction is consistency with pathological results. All patients enrolled in the study were classified according to the 2021 WHO (World Health Organization) pathological classification [14]. Two experienced neuropathologists, XL and YJH, each with 10 years of experience in neuropathology, independently reviewed the histological diagnoses for both the development and independent test datasets. Their reviews were aligned with the 2021 WHO classification of tumors of the central nervous system [15]. Any specimens that did not conform to the 2021 WHO classification underwent a secondary review by other neuropathologists (JD with 30 years of experience). The criteria for immunohistochemical and diagnostic assessments in the independent validation dataset were consistent with those used for the development dataset.

The quality of facial photographs was evaluated by 2 investigators who were blinded to the study design, as outlined in the protocol (Method S2 in Multimedia Appendix 1.). Those that were deemed qualified were processed by cropping to isolate only the facial area, thus removing any excess background and clothing, and were resized to the dimensions of 256×256 pixels. Figure 2 depicts the ResNet-50 model workflow for facial photo analysis, including preprocessing, configuration, training with loss function and optimizer selection, and evaluation using receiver operating characteristic (ROC) and precision-recall curves on an independent external test dataset.

For our statistical analysis, we used SPSS (version 25.0) software (IBM Corp). Continuous variables were presented as mean (SD), and discrete variables as percentages. We assessed the normality of continuous variables using the Shapiro-Wilk test. Depending on the results, either the independent samples t test, Friedman test, or the Mann-Whitney U test was used for comparison. Categorical variables, such as gender and endocrine abnormalities in patients with NCs and midline brain tumors (including iGCTs and other midline brain tumors), were analyzed using the chi-square test. We calculated the 95% exact CIs for both proportions and mean differences.

To evaluate the performance of the DL model, we computed its diagnostic accuracy, sensitivity, specificity, and the area under the ROC curve, using pathological diagnosis as the benchmark. The model’s performance was evaluated at 2 specific points on the receiver operating characteristic ROC curve, chosen for their maximum sensitivity and specificity. We used the Delong test [18] to compare the AUCs of the different models. Multinomial logistic regression was used to examine the correlation between the prediction scores and the endocrine indicators. All comparisons were conducted as 2-sided tests, with a P value of less than .05 considered statistically significant.

Ethical approval was granted by the institutional review board of Beijing Tiantan Hospital, Capital Medical University (KY2021-142-02). Informed consent was secured from all patients or their guardians. Privacy and Confidentiality: In this study, all participant data, including images, are processed with strict measures to ensure privacy and confidentiality. Each participant’s data, including facial photographs, is anonymized and deidentified before analysis to prevent any direct or indirect identification. For cases where anonymization is not feasible, such as if the integrity of facial features is required for model training or analysis, robust data protection protocols are implemented. These include secure storage solutions, restricted access, and ethical review board oversight, ensuring compliance with data privacy standards.

Compensation details are that the participants involved in the study were compensated fairly for their contributions. All individuals provided informed consent, detailing the nature and amount of compensation received, proportional to the time and effort required for their participation, in accordance with ethical guidelines.

Initially, we obtained frontal facial photographs of 989 subjects (including 300 NCs, 437 patients with iGCTs, and 252 patients with other midline brain tumors). These photos were collected for the training and testing of the DL model with facial recognition, GVisageNet, from January 2010 to December 2020 at 2 medical institutes. In addition, an independent validation dataset, comprising 200 NCs, 130 patients with iGCTs, and 106 patients with other midline brain tumors, was collected from four medical institutes between January 2021 and May 2023 for validation of the DL model. The demographic data, symptoms, and endocrine hormone levels of all patients across all datasets (training, testing, and independent validation) are summarized in Table 1. The training, testing and independent validation datasets included 500 NCs (250 males and 250 females, with mean age of 13.00, SD 5.15 years), 567 patients with iGCTs (256 males and 290 females, with a mean age of 12.81, SD 5.79 years) and 358 patients with other midline brain tumors 155 males and 203 females, with a mean age of 13.61, SD 4.93 years), respectively. We filtered 7 clinical variables (limb movement disorder, diabetes insipidus, sexual abnormality, affective disturbance, convulsive seizure, TSH, and GH) that were statistically different between the iGCTs group and other midline brain tumors group in the training/testing dataset. Figures 3A and 3B illustrate the distribution of iGCT tumor sites across various hospitals in different datasets, as well as the distribution of tumor types within all datasets among other midline brain tumor groups.

In this investigation, we conducted a comparative analysis of 3 distinct candidate algorithms by training each on the training dataset and then evaluating their respective performances on the testing dataset. The algorithms tested were AlexNet, ResNet-50, and Inception. In the testing dataset, GVisageNet and the other models (AlexNet and Inception) yielded similar results in the first stage (distinguishing patients with midline brain tumors and NCs) with AUCs of 0.972 (95% CI 0.951-0.992), 0.963 (95% CI 0.943-0.984), and 0.983 (95% CI 0.937-1.000), respectively, and P>.05. Therefore, we proceeded to compare the results from the second stage. In the second stage, based on the AUC values with 95% CIs, the following results were observed: AlexNet achieved an AUC of 0.667 (95% CI 0.582-0.777), ResNet-50 reached an AUC of 0.705 (95% CI 0.590-0.760), and Inception yielded an AUC of 0.657 (95% CI: 0.583-0.776; Figure 3C and Table 2). Given these outcomes, ResNet-50 emerged as the most proficient model in terms of classification performance and was subsequently chosen as the final model for our classification tasks. Table S1 in Multimedia Appendix 1 and Figure S2 in Multimedia Appendix 1 presented additional details.

In the testing dataset, GVisageNet achieved an AUC of 0.942 (95% CI 0.921-0.972), an accuracy of 94.33% (95% CI: 91.00%-95.96%), a sensitivity of 93.51% (95% CI: 90.00%-96.50%), and a specificity of 93.39% (95% CI 89.48%-97.00%) in distinguishing NCs from midline brain tumors. In second stage, GVisageNet achieved an AUC of 0.705 (95% CI 0.590-0.760), an accuracy of 71.71% (95% CI 57.42%-77.11%), a sensitivity of 75.24% (95% CI 70.89%-77.57%), and a specificity of 73.40% (95% CI 69.04%-76.80%) in distinguishing iGCTs from other midline brain tumors.

In the independent external validation dataset, GVisageNet demonstrated effectiveness in distinguishing NCs and midline brain tumors. It achieved an AUC of 0.938 (95% CI 0.913-0.967), an accuracy of 83.77% (95% CI: 71.45%-92.33%), a sensitivity of 91.67% (95% CI 87.42%-100.00%), and a specificity of 77.55% (95% CI 66.43%-80.46%; Figure 4A). In distinguishing iGCTs from other midline brain tumors, as shown in Figure 4B, it achieved an AUC of 0.739 (95% CI 0.728-0.773), an accuracy of 73.10% (95% CI 69.85%-77.34%), a sensitivity of 68.33% (95% CI 53.65%-75.14%), and a specificity of 63.24% (95% CI 56.59%-81.20%). The AUC, accuracy, sensitivity, and specificity of GVisageNet in the testing and validation datasets of the second stage are depicted in Figures 4C and 4D.

To enhance GvisageNet’s ability to distinguish iGCTs from other midline brain tumors, we compared the performance of a logistic regression model using clinical variables with a hybrid model. In the testing dataset, the logistic regression model, which used clinical variables, achieved an AUC of 0.612 (95% CI 0.493-0.709), an accuracy of 59.41% (95% CI 50.85%-64.28%), a sensitivity of 65.18% (95% CI 56.78%-73.54%), and a specificity of 55.93% (95% CI 48.48%-65.74%). In the independent external validation dataset, this model attained an AUC of 0.632 (95% CI 0.571-0.679), an accuracy of 68.97% (95% CI 62.68%-75.44%), a sensitivity of 57.88% (95% CI 49.45%-68.74%), and a specificity of 58.59% (95% CI: 49.24%-70.04%), as depicted in Figures 4C and 4D, and Table S2 in Multimedia Appendix 1.

The hybrid model in the testing dataset achieved an AUC of 0.711 (95% CI 0.676-0.798), an accuracy of 70.77% (95% CI 65.48%-76.65%), a sensitivity of 72.94% (95% CI 66.99%-76.84%), and a specificity of 67.78% (95% CI 66.31%-72.53%). In the independent external validation dataset, the hybrid prediction model reached an AUC of 0.789 (95% CI 0.728-0.793), an accuracy of 71.56% (95% CI 68.90%-73.84%), a sensitivity of 71.77% (95% CI 56.48%-76.88%), and a specificity of 58.45% (95% CI 50.73%-66.05%), as shown in Figures 4C and 4D.

In a comparative analysis conducted on the independent external validation dataset, GVisageNet was found to significantly outperform the logistic regression model, demonstrating higher accuracy (AUC 0.739 vs 0.632; P<.001). However, its performance was slightly inferior to that of the hybrid model (AUC 0.739 vs 0.789; P=.04).

Correlation analysis revealed a significant positive correlation between the prediction scores and the levels of TSH and IGFBP-3 (P<.05). Conversely, there was a significant negative correlation between the prediction scores and the levels of total thyroxine (TT4), FT3, FT4, GH, prolactin, and estradiol (P<.05). Detailed pairwise concordances of endocrine indicators are presented in Figure 5A.

Moreover, the independent validation dataset was divided into 3 groups based on the degree of HPT axis damage: complete damage (n=37), incomplete damage (n=54), and no damage (n=39). In the task of differentiating iGCTs from other midline brain tumors, GVisageNet achieved varying levels of performance across these groups. In the complete damage group, GVisageNet achieved an AUC of 0.678 (95% CI 0.443-0.717), an accuracy of 67.89% (95% CI 57.79%-76.33%), a sensitivity of 63.21% (95% CI 57.67%-75.54%), and a specificity of 57.58% (95% CI 47.99%-74.91%). For the incomplete damage group, the model reached an AUC of 0.575 (95% CI 0.412-0.674), an accuracy of 55.43% (95% CI 35.37%-68.54%), a sensitivity of 59.56% (95% CI 46.43%-76.54%), and a specificity of 53.54% (95% CI 39.54%-66.64%). In the no-damage group, GVisageNet achieved an AUC of 0.517 (95% CI 0.412-0.774), an accuracy of 44.32% (95% CI 38.43%-77.33%), a sensitivity of 59.54% (95% CI 45.43%-66.47%), and a specificity of 48.11% (95% CI 33.23%-68.42%; Table 3). The AUC, accuracy, sensitivity, and specificity for each group are detailed in Figure 5B. An analysis of variance (Friedman test) revealed a significant difference in AUC among the 3 groups (P=.03). The visual integration of GVisageNet into the facial photo collection system is shown in Figure 5C.

This study successfully developed and validated an automated binary DL model, GVisageNet, to differentiate midline brain tumors from NCs and further distinguish iGCTs from other midline brain tumors using facial images. We also constructed a logistic regression model and a hybrid model using clinical data to explore whether clinical variables could enhance the diagnostic performance of the model. The results showed that in differentiating iGCTs from other midline brain tumors, the hybrid model slightly outperformed GVisageNet (0.739 vs 0.789; P=.04). Finally, we attempted to explain the mechanism of GVisageNet by analyzing endocrine changes in patients and found that the model’s predicted probability values were strongly correlated with seven endocrine indicators (P<.05). In addition, GVisageNet performed best in the group with complete endocrine axis damage, suggesting that endocrine axis impairment may be a reason for the facial texture changes observed in patients. GVisageNet can be integrated into outpatient facial photo collection systems, enabling the routine collection of facial images during a patient's initial visit. This allows for real-time preliminary screening diagnoses, guiding clinicians to conduct targeted examinations, thereby saving medical resources and reducing patient costs.

The enhanced ResNet-50 model demonstrated significant advancements in this field, outperforming both its original version and other conventional CNN models in terms of training and testing accuracy. This improvement was achieved through structural modifications and an optimized training process, including an adaptive learning rate that effectively adjusts weights within the network layers. Moreover, the model effectively addressed the common issue of overfitting in DL, thereby enhancing its reliability and consistency in screening applications. A crucial aspect of this improvement was the model’s reduced loss values and fluctuations, contributing to its increased stability. Importantly, visualization tools were used for feature extraction from specific layers of ResNet-50 [19], enabling precise identification of potential tumor indicators from facial photographs [20], which is essential for early screening and intervention. In addition, the introduction of a standard operating procedure for preprocessing facial photos significantly improved the quality of input data, aiding in more accurate tumor screening.

In addition to the analyses conducted, it is essential to consider the potential issue of collinearity among features. Collinearity, or high intercorrelation among predictor variables, can adversely affect the performance and interpretability of regression models and other machine learning techniques. Detecting and addressing collinearity is crucial to ensure the robustness of our models. Methods such as correlation matrices, variance inflation factors, and condition indices can be used to detect collinearity [21]. Mitigation strategies include removing or combining highly correlated features and using regularization techniques like Ridge regression or Lasso [22]. Future work should include a thorough investigation of collinearity and its impact on model performance. By addressing collinearity, we can enhance the stability and interpretability of our models, leading to more reliable diagnostic predictions. Therefore, it is recommended that future studies incorporate collinearity diagnostics and report any steps taken to mitigate its effects. This approach will provide a more comprehensive understanding of the model development process and ensure the robustness of the predictive models.

Furthermore, DL models, especially those designed for facial recognition, are adept at extracting and analyzing complex image patterns that may reflect physiological changes due to endocrine imbalances. Several mechanisms underlie the potential correlations between DL model outputs and endocrine indicators. First, models like CNNs can identify subtle changes in facial features associated with hormonal imbalances, such as those caused by conditions like acromegaly or Cushing’s syndrome, which alter fat distribution, skin texture, or facial bone structure. Second, these models use extensive training datasets to recognize patterns associated with specific conditions, such as the protruding eyes or facial puffiness seen in hyperthyroidism, and can correlate these visual cues with hormone levels (eg, TSH, triiodothyronine, and thyroxine) to predict endocrine states from the facial analysis. In addition, hormones significantly affect physical characteristics, where elevated cortisol levels may cause a “moon face” appearance or a buffalo hump, and excess growth hormone can change the jawline and brow prominence, all detectable through DL analysis. Finally, integrating DL outputs with clinical data, such as hormone levels, enhances predictive accuracy. A hybrid model that merges image-based phenotypic features with biochemical markers provides a comprehensive understanding of a patient’s endocrine state, enabling nuanced predictions and personalized health assessments. This integrated approach leverages facial recognition technology to noninvasively assess endocrine conditions, thereby enhancing diagnostic precision and patient care.

In addition, exploring the potential broader applications of our DL model in diagnosing other diseases with distinct phenotypic manifestations is imperative. For instance, conditions such as Parkinson disease, characterized by specific facial expressions and muscle rigidity, or genetic disorders like Down syndrome, which presents with unique facial features, could benefit from analogous DL-based diagnostic approaches [23,24]. By training models to recognize these specific phenotypic patterns, we can expand the use of our DL model to provide early and accurate diagnoses across a variety of medical conditions. This broader application not only enhances the model’s relevance in the medical field but also contributes to more comprehensive patient care through the early detection of diseases that manifest through observable physical changes.

Significant correlations were observed between DL model outputs and endocrine markers, including TSH, IGFBP-3, TT4, FT3, FT4, GH, prolactin, and estradiol, suggesting that endocrine variations may play a crucial role in the development of facial features in patents with iGCT. Notably, approximately 40% of iGCTs secrete β-HCG [1,5,25], which also induces endocrine changes at sufficiently high levels. The cross-immunoreactivity of β-HCG with LH triggers peripheral precocious puberty or gonadotropin-releasing hormone–independent precocious puberty, distinct from central precocious puberty and nonprogressive pubertal variants. In males, elevated β-HCG levels activate LH receptors on Leydig cells, enhancing testosterone production [26]. Conversely, in females, β-HCG does not induce precocious puberty, as activation of both FSH and LH receptors is requisite. Consequently, precocious puberty in iGCTs is predominantly observed in male toddlers and school-aged boys, characterized by facial features such as acne, beard growth, increased oil secretion, and skeletal maturity. Excess GH production leads to acromegaly [27], resulting in enlarged facial bones, while hypothyroidism causes facial puffiness [28-32], especially around the eyes [33-35]. When DL model results were integrated with clinical indicators, the detection efficacy was significantly enhanced. It is imperative to clarify that β-HCG was excluded from our study since only 40% of patients exhibit β-HCG secretion, and the DL model serves as a prediagnostic screening tool, hinting at further examination. Premature inclusion of tumor markers could compromise model accuracy. This observation was further underscored in the stratified analysis. Our findings indicate that the predictive efficacy of GVisageNet was least effective in cases without HPT axis impairment, and most effective in instances of complete HPT axis disruption. This underscores a direct correlation between the model's predictive capacity and endocrine function. Notably, when the HPT axis was fully compromised, the resultant endocrine-driven alterations in facial characteristics were most pronounced.

This study had several limitations that should be considered when interpreting the findings. First, the relatively small sample size of each iGCT subtype (germ cell tumors [germinomas] and nongerminomatous germ cell tumors) limited the ability to conduct a more detailed analysis and might have impacted the statistical power of the study. Future research should aim to include larger cohorts to enhance the robustness and generalizability of the findings. Second, the study only collected baseline facial photos of the patients. The lack of longitudinal follow-up data limits the ability to observe changes in facial features and endocrine parameters over time. Longitudinal studies are needed to evaluate the progression and potential predictive value of facial recognition technology in monitoring patients with iGCTs. Third, the study was conducted across multiple centers, which introduced variability in data collection and image quality. Although efforts were made to standardize procedures, some degree of inconsistency is inevitable. Future studies should implement stricter protocols and quality control measures to minimize such variability. Finally, while the study focused on differentiating iGCTs from other midline brain tumors, it did not explore the potential of this technology in distinguishing iGCTs from other types of intracranial tumors or nontumorous conditions. Expanding the scope of future research could provide a more comprehensive understanding of the clinical applications of deep learning-based facial recognition in neuro-oncology.

In addition, “no identifiable personal information” (such as unique facial characteristics) is presented in any figures or supplementary materials of the manuscript. In cases where participant consent is necessary for image inclusion, signed consent forms have been obtained and can be uploaded in the “Other files” field upon resubmission. This ensures transparency and adheres to the highest standards of ethical research practices.

In summary, this study demonstrates the potential application of facial recognition technology in the diagnosis of iGCTs. Facial recognition could not only aid in early detection and screening of iGCTs but also might become a valuable tool in advancing future clinical decision-making.