A Multidimensional Comparative Study of Multiple Mainstream Large Language Models in Perioperative Consultation for Hypospadias Surgery
Date Submitted: Feb 12, 2026
Open Peer Review Period: Feb 13, 2026 - Apr 10, 2026
Background: Hypospadias is a common congenital malformation requiring surgical correction, with caregivers facing significant perioperative information needs. Large language models (LLMs) offer a potential solution for health education, yet their performance in pediatric urological consultations remains unexplored. Objective: This study aimed to evaluate the performance of five LLMs—ChatGPT-4o, Gemini-2.5-Pro, OpenEvidence, Zhipu Qingyan, and DeepSeek—in addressing core concerns of caregivers during the perioperative period for children with hypospadias, and to assess their application value and limitations in pediatric urological clinical health education. Methods: A prospective, non-interventional, cross-sectional design was employed. A question bank was developed based on literature and clinical practice, with 10 high-priority questions selected via questionnaire screening for the test set. Seven experts (6 dimensions) and 32 caregivers (4 dimensions) were prospectively recruited to evaluate responses using a double-blind forced-order ranking method (reverse scoring from 1 to 5), with reference authenticity being verified. Results: Significant differences existed across dimensions among the five models (P<.001). Gemini-2.5-Pro demonstrated overall superior performance, ranking first in both expert evaluations (median 5.0 [IQR 4.0-5.0]) and caregiver evaluations (median 4.0 [IQR 3.0-5.0]), with outstanding structural capabilities; DeepSeek ranked second (median 4.0), demonstrating relatively consistent ratings across socioeconomic strata and superior emotional support compared to ChatGPT-4o (P=.001); OpenEvidence scored lowest (nearly 50% “poor” ratings), exhibiting poor readability but reliable evidence sources. Conclusions: Gemini-2.5-Pro offers the most comprehensive quality for perioperative hypospadias consultations. DeepSeek and Zhipu Qingyan demonstrate strong rapport in home care guidance but require strict control of literature hallucination risks. Perioperative care for hypospadias should adopt a tiered human-machine collaboration model based on clinical risk stratification, balancing communication efficiency and health education safety.
