This study is reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis-AI (TRIPOD-AI) guidelines. The study included adult patients (≥18 years) who were scheduled for surgical procedures between August 26, 2024, and May 5, 2025, across a range of surgical specialties at UAMS (Textbox 1).

Inclusion criteria included a documented ASA-PS score assigned by the attending anesthesiologist, at least 1 preoperative clinical note dated prior to the date of procedure, and structured clinical data with less than 95% missingness. Exclusion criteria included absence of any preoperative clinical notes, missing ASA-PS scores, structured data missingness greater than 95%, emergency cases, and canceled procedures.

From the total cohort meeting eligibility criteria, 3000 encounters were included. Demographic features between the training, validation, and testing cohorts were compared to ensure the data split did not introduce confounding imbalances (Table 1). ASA-PS 5 cases, representing moribund patients, are rarely encountered in elective preoperative settings and were correspondingly rare here; their scarcity renders class-level estimates unstable at this sample size and limits their relevance to the elective use case.

Age was compared using an independent 2-sample t test. Gender and language were assessed using chi-square tests. Race and ethnicity data were not available and therefore could not be evaluated. No prior sample size or power calculation was performed. Instead, the study included all eligible cases within the retrospective study period (between August 26, 2024, and May 5, 2025).

Only information documented prior to the scheduled surgical start time was used as model input to best approximate the ASA-PS score assigned by a day-of-surgery anesthesiologist. Structured data comprised more than 400 preoperative variables, including demographics, comorbidities, laboratory values, vital signs, medications, and procedural metadata. Language and gender were excluded from model inputs due to the risk that their inclusion could encode systemic documentation inequities and rater bias into ASA-PS predictions rather than true clinical signal [23-26]. Age was retained as a clinically validated predictor of comorbidity burden and perioperative risk, consistent with its established role across surgical risk stratification literature [27]. Incarceration status was retained under the same rationale. Unlike gender, language, race, and ethnicity, whose associations with assigned ASA-PS plausibly reflect documentation practices and rater perception rather than incremental clinical information, incarceration is associated with systematically elevated burdens of chronic, infectious, psychiatric, and substance use disease and with disrupted access to longitudinal care that are incompletely captured by structured comorbidity fields [28,29]. It was therefore treated as a marker of clinical risk exposure, analogous to age, rather than as a demographic descriptor. Requests for structured data components can be directed to the corresponding author (AF). Across the 3000 included encounters, median structured data missingness was 57.6% (IQR 38.8%‐92.5%). This sparsity primarily reflects the wide-pivoted feature representation, in which each column encodes a specific clinical concept and disease-specific variables are legitimately null for lower-acuity patients, rather than data-quality failure. Numeric missingness was mean-imputed using an imputer fit on the training data, while categorical missingness was retained as an explicit level; a stricter missingness threshold would have preferentially excluded the lower-acuity majority.

Medical diagnoses were captured through 2 complementary approaches: (1) direct extraction from electronic health record problem lists and International Statistical Classification of Diseases, Tenth Revision (ICD-10) codes, and (2) a proprietary LLM pipeline that analyzed free-text notes to identify additional medical diagnoses not recorded in structured fields. Unstructured notes were filtered to 50 curated note types, including progress notes, history and physical, discharge summaries, operative notes, anesthesia evaluations, emergency department notes, consult notes, and nursing notes. Duplicates were deduplicated on patient ID and note ID, keeping the most recent version. All notes per patient were concatenated and truncated to 100,000 tokens, preserving the latest content. A 3-year lookback window was applied. No further text cleaning was performed; raw plain text was preserved to retain clinical nuance.

From the total cohort of 3000 encounters, cases were randomly allocated to a training set (n=2000), a validation set (n=500), and a held-out test set (n=500). Cases were assigned to training, validation, and test sets using stratified random sampling based on documented ASA-PS scores, to ensure balanced representation of ASA-PS classes across all 3 sets.

For all model training and final testing, the day-of-procedure anesthesiologist-assigned ASA-PS served as the ground truth. Each case reflects the assessment of a single attending anesthesiologist as documented in the electronic health record; no multirater adjudication or consensus process was applied to the reference labels. This approach was selected to follow established literature [11-14,16] for automated ASA-PS classification predictions and to ensure the model is assessed against the real-world clinical benchmark, including the inherent label uncertainty introduced by known interrater variability.

Three distinct LLM prompts were used, each serving a separate functional role: (1) clinical diagnosis extraction from unstructured notes, (2) ASA-PS classification, and (3) generation of structured ASA-PS explanations for a subset of examined architectures. All 3 were implemented using GPT-4o queried at temperature=0 to ensure deterministic outputs, with concatenated patient notes truncated to 100,000 tokens prior to submission and responses returned in structured JSON format.

The clinical diagnosis extraction prompt analyzed free-text notes to identify medical diagnoses not recorded in structured electronic health record fields, supplementing the problem-list and ICD-10 data described above. Extraction performance was validated in a sample of 221 UAMS patients, separate from the 3,000-encounter model development cohort. For each patient, a predefined set of 168 candidate clinical facts was evaluated, yielding 37,128 patient-fact units. The reference standard was established by 2 board-certified anesthesiologists, who each reviewed the anesthesia preprocedure evaluation for all 221 patients and determined which of the 168 facts were captured in that documentation. The extraction pipeline is internally validated at a sensitivity of 0.738 (95% CI 0.719‐0.752), precision of 0.845 (95% CI 0.832‐0.857), and specificity of 0.978 (95% CI 0.976‐0.980), and its outputs were folded into the structured feature set supplied to the ML models. CIs were estimated by bootstrap resampling at the patient level to account for the nonindependence of facts within a patient.

The ASA-PS classification prompt was used to generate the ASA-PS prediction in the LLM-only baseline and as the LLM stream within the hybrid architectures. This prompt was structured around 3 sequential modules: clinical information extraction, severity grading, and ASA-PS class assignment. The model was instructed to reason through each module before producing a final output, encouraging consistent intermediate reasoning analogous to the regularization effect of multitask learning. It was anchored to the ASA 2020 Physical Status criteria, which informed the threshold-based decision logic embedded in the severity-grading module, so that the model performed criteria-anchored reasoning rather than relying on pretraining exposure to ASA-PS language alone. Each response contained a final ASA-PS prediction, binary indicators for ASA-PS classes 1-5, a brief clinical rationale, and a confidence score; because temperature was fixed at 0, top-p (nucleus) sampling was nonoperative, and max_tokens was left unconstrained (responses averaged well under 500 tokens). The complete template, including its scaffolded reasoning structure and embedded decision logic, is available to qualified investigators upon request to the corresponding author AF.

A third ASA-PS explanation prompt was used in the Sequential Hybrid (with and without embeddings) and Parallel Ensemble architectures. This prompt generated structured explanations of each score, outlining the key comorbidities, severity descriptors, and functional-status indicators that contributed to it. These explanations were constrained to a standardized format to enable consistent interpretation and independent adjudication of the model’s reasoning. Because XGBoost is the terminal step in the LLM-as-Features alternative, no LLM-guided explanation was generated in that pipeline.

The XGBoost, Light Gradient Boosting Machine (LightGBM), and ExtraTrees models were trained using a standardized pipeline with stratified 3-fold cross-validation and Optuna-based (Bayesian) hyperparameter optimization. A shared preprocessing pipeline was applied to the structured feature set before model fitting, so that all 3 classifiers received identical inputs. Features with ≥95% missingness were dropped. For categorical features, missing values were retained as an explicit “missing” level prior to label encoding, preserving nonrecording as potentially informative. Remaining numeric features were mean-imputed using an imputer fit on the training data alone and applied unchanged to the validation and held-out test sets, preventing test-set information from entering the imputation statistics. Random oversampling of minority ASA-PS classes was then applied to the training data only, after imputation, so that no imputation statistic was computed from duplicated rows. Validation and held-out test sets retained their native class distributions. The objective was to maximize macro-averaged F₁-score across ASA-PS classes, with stratified folds preserving the class distribution within each cross-validation split. For each model, the dataset was split into 3 stratified folds, with Optuna tuning performed within each fold; key hyperparameters (eg, tree depth, learning rate, and regularization strength) were searched within predefined ranges. The mean macro-F₁-score across folds was used to select optimal hyperparameters, and final models were retrained on the full training set and evaluated on the validation set. All stochastic operations (cross-validation splits, Optuna sampling, and final model training) used a fixed random seed (42) to ensure reproducibility. An LLM-only baseline used GPT-4o to predict ASA-PS scores from preoperative notes alone, applying the ASA-PS classification prompt described above. Notes-only input consistently outperformed configurations that added structured data, so the final pipeline used text alone.

Hybrid architecture development is depicted in Figure 1.

In the Sequential Hybrid with Embeddings architecture (Figure 1A), structured data were combined with 768-dimensional Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) embeddings of patient notes to create an augmented feature vector, which was used to train an XGBoost classifier. A confidence gating mechanism routed high-confidence cases (≥95%) to XGBoost directly, while lower-confidence cases were forwarded to GPT-4o for final classification with access to notes, structured data, and the ML prediction (Figure 1). Performance was assessed using macro-F₁-score and accuracy, with coverage analysis to quantify the proportion of cases handled by ML versus LLM. All evaluations were conducted on the validation set.

In the Sequential Hybrid without Embeddings architecture (Figure 1B), structured data were used to train an XGBoost classifier, with a confidence gating mechanism routing high-confidence cases (≥95%) to XGBoost and low-confidence cases to GPT-4o. GPT-4o received the original notes, structured data summary, and ML prediction. Performance was assessed using the same metrics as the embedding pipeline to isolate the effect of excluding ClinicalBERT embeddings, with additional analysis of confidence coverage and anchoring bias. XGBoost output probabilities were not formally calibrated (eg, via Isotonic Regression or Platt Scaling) prior to applying the confidence gating threshold. While preliminary internal review suggested calibration did not substantially alter routing behavior or downstream performance, formal prospective calibration using out-of-fold training predictions is recommended before clinical deployment.

In the Parallel Ensemble architecture (Figure 1C), structured data were processed by XGBoost and clinical notes by GPT-4o in fully independent streams to avoid the anchoring effects of sequential designs. Final predictions were generated through weighted averaging of the ML probability distribution and LLM classification (optimal α=.30, corresponding to 30% ML and 70% LLM). Performance was evaluated against individual components using macro-F₁, accuracy, and calibration metrics, with sensitivity analysis across weighting values.

In the LLM-as-Features Alternative (Figure 1D), GPT-4o was first prompted to generate ASA-PS predictions and intermediate markers (binary flags for ASA-PS classes 1-5) from clinical notes. These outputs were concatenated with the 400+ structured clinical variables to form an augmented feature set, which was used to train a second XGBoost model. This allowed the algorithm to learn optimal weighting between structured data and LLM-derived features. The same structured feature set and the same preprocessing pipeline described above were used across all hybrid model architectures; no features were added or removed between the sequential and parallel configurations. Structured variables were drawn from 8 source categories: chart-mined clinical facts, ICD-10 condition matrices, case-level features (eg, surgical urgency and procedure risk), 34 laboratory value types, 9 vital sign types, device flags, medication flags, and age.

Feature importance was quantified via Shapley additive explanations (SHAP) TreeExplainer on the XGBoost component of the parallel ensemble. Per-class SHAP summary plots, showing the direction of each feature’s contribution to every ASA-PS class, are provided in Multimedia Appendix 1.

Model performance was evaluated using macro-averaged F₁, exact-match accuracy, and ±1-class accuracy. Ninety-five percent CIs were estimated by bootstrap resampling (1000 iterations), and pairwise model comparisons used permutation testing (1000 permutations), with P<.05 considered statistically significant. Analyses were performed in Python (Python Software Foundation, Wilmington, DE, United States); package versions for XGBoost, LightGBM, scikit-learn, Optuna, and SHAP, along with the exact GPT-4o snapshot and the ClinicalBERT variant, were used for the 768-dimensional embeddings.

To quantify the inherent variability in provider interpretation of the ASA-PS classification system and establish a local performance ceiling, we conducted an internal reliability study independent of all automated pipelines. Two board-certified anesthesiologists (AF, GF) independently reviewed a subset of 50 randomly selected cases drawn from the broader eligible cohort, separate from the validation and held-out test sets, and assigned ASA-PS scores based solely on preoperative documentation. Agreement for each pairwise comparison was quantified as macro-F₁.

Both reviewers maintain active clinical responsibilities that include preoperative patient management and are external to the source institution, providing independent perspective and guarding against local anchoring bias. Because these cases were sequestered from all model development and evaluation sets, the resulting ceiling did not influence hyperparameter tuning, architecture selection, or alpha weighting.

The expert ceiling was computed from 3 rating sources for each of the 50 cases: the 2 panel anesthesiologists and the day-of-surgery attending anesthesiologist whose documented ASA-PS served as the reference standard throughout this study. Three pairwise comparisons are defined among these 3 sources: panel anesthesiologist 1 versus the day-of-surgery attending, panel anesthesiologist 2 versus the day-of-surgery attending, and panel anesthesiologist 1 versus panel anesthesiologist 2. The ceiling of 0.59 is the mean macro-F₁-score across these 3 comparisons. Macro-F₁-score is symmetric under exchange of the 2 rating sets, so the panel-versus-panel comparison does not require designating either rater as the reference. Each comparison was computed on the cases for which both members of the pair provided a rating. This mean reflects both alignment with the clinical reference standard and interreviewer concordance, and is consistent with prior literature demonstrating moderate interrater reliability for ASA-PS scoring (κ ≈ 0.4‐0.6), where perfect alignment is neither clinically expected nor mathematically attainable. Three rating sources across 50 cases yield 150 possible ratings, of which 137 were available: all 50 day-of-surgery ratings, present by inclusion criteria, and 87 of 100 possible panel ratings, with 13 omitted because insufficient documentation precluded expert classification. The per-class distribution of all 137 ratings was ASA-PS 1 (n=17), ASA-PS 2 (n=31), ASA-PS 3 (n=52), ASA-PS 4 (n=33), and ASA-PS 5 (n=4).

This retrospective observational study was approved by the Institutional Review Board (IRB) of the University of Arkansas for Medical Sciences (UAMS). Given the retrospective nature of the study and use of existing clinical data, the IRB granted a waiver of informed consent (IRB number 298590); the original clinical data were collected under standard institutional consent practices that permit secondary analysis. Direct identifiers were not retained in the analytic dataset, and all coded data were stored in a secure, Health Insurance Portability and Accountability Act (HIPAA)-compliant Amazon Web Services environment maintained by Qventus under a Business Associate Agreement. No compensation was provided to study participants. No identifiable patient images are included in this paper or Multimedia Appendix 1.

Among the evaluated configurations, the parallel ensemble (α=.30) was identified as the optimal architecture for ASA-PS classification. On the held-out test set, the parallel ensemble achieved a macro-F₁-score of 0.530 (95% CI 0.350‐0.600), exact match accuracy of 0.630 (95% CI 0.580‐0.670), ±1-class accuracy of 0.980 (95% CI 0.968‐0.992), and Cohen κ of 0.34, confirming generalizability beyond the development cohort. On the validation set, the standalone LLM-only baseline achieved a macro-F₁-score of 0.642 (95% CI 0.525‐0.691), nominally exceeding the expert panel ceiling of 0.59 (derived as the average macro-F₁-score across 3 pairwise comparisons: each of 2 board-certified anesthesiologists compared against the day-of-surgery reference standard, and the 2 anesthesiologists compared with each other). This result requires careful interpretation, as it was not reproduced when the same frozen baseline was applied to the held-out test set, where the LLM-only architecture achieved a macro-F₁-score of 0.493 (95% CI 0.368‐0.585). Per-class analysis localizes this discrepancy to a single class: ASA-PS V performance on the validation set (n=2, F₁-score=1.00) mechanically inflates the unweighted macro average, a known artifact of macro-averaging under extreme class rarity, and this inflation is not reproduced on the held-out set. Excluding ASA-PS V, the remaining 4 validation classes yield a macro-F₁-score of approximately 0.55, closely aligned with the parallel ensemble and within the expert interrater reference range. The held-out estimate of 0.493 is a substantially closer match to this ASA-PS V-excluded validation figure than to the nominal 0.642, and the held-out result should therefore be regarded as the more faithful representation of broader LLM-only performance. The apparent validation advantage reflects the arithmetic of 2 correctly classified ASA-PS V cases rather than a generalizable superiority over the expert ceiling.

LLM consistency and human validity are distinct properties: given documented interrater reliability of 0.4‐0.6 in ASA-PS classification, it is plausible that GPT-4o achieves greater internal consistency than individual raters. However, consistency with a human-labeled standard does not constitute clinical superiority, particularly when concentrated in 2 extreme-acuity cases. Convergence of LLM-only performance toward the parallel ensemble on the true test set further supports this interpretation. Pairwise permutation testing confirmed that the parallel ensemble and LLM-only baseline were statistically indistinguishable on macro-F₁-score and exact match accuracy (P>.99). Model selection was therefore guided by architectural considerations: by independently integrating objective structured clinical data alongside LLM-derived narrative interpretation, the parallel ensemble anchors predictions to measurable clinical variables in a way an LLM-only approach cannot. This is particularly important for minority classes where structured indicators (eg, laboratory values, medication burden, and documented comorbidities) may diverge from narrative tone.

The test-set Cohen κ of 0.34 warrants direct consideration against the 0.4‐0.6 interrater reliability cited in the Introduction section. Published interrater coefficients are derived from controlled studies in which multiple raters score identical cases, whereas our kappa is computed against single-rater-day-of-surgery labels that already embed the full spectrum of interrater variability, so the model is evaluated against a moving target that human benchmarks never face. Kappa and macro-F₁-score also capture different properties: kappa corrects for chance agreement and is sensitive to the ASA-PS II-III predominance of this cohort, mechanically depressing it relative to macro-F₁-score. The case for deployment therefore rests not on matching expert kappa but on the 98.0% ±1-class accuracy, which captures the clinically operative distinction between acceptable and dangerous misclassification, together with the model’s capacity for consistent, auditable, and explainable classification at scale.

Due to the ASA-PS classification’s basis on subjective clinical judgment, the ideal state for any model to be measured against is itself imperfect. Rather than targeting perfect agreement with reference labels, a more meaningful ideal for an automated classification system would be ±1-class accuracy approaching 100% with a macro-F₁-score approaching, though not necessarily exceeding, the expert panel’s benchmark. The parallel ensemble’s exact match accuracy of 63% (95% CI 0.580‐0.670) on the held-out test set aligns with the documented limitations of ASA-PS classification itself. Our model’s performance demonstrates effective extraction of signal from inherently subjective clinical data, approaching the limits imposed by the classification system’s fundamental ambiguity.

The parallel ensemble model’s 98.0% accuracy within one ASA-PS class (95% CI 0.968‐0.992) on the held-out test set serves as a critical safety mechanism. Extreme misclassifications (errors of 2 or more classes) in perioperative medicine pose a great risk to patient safety, potentially routing high-acuity patients to inadequate care before procedures or inadequately resourced surgical venues. By heavily reducing these outliers, the parallel ensemble ensures automated classifications remain within clinically acceptable bounds. While exact match accuracy remains the ideal, this containment of extreme errors provides the operational stability necessary for safe implementation across diverse perioperative settings.

The clinical consequences of misclassification are not uniform across the ASA-PS rank order. Errors at the lower end, between ASA-PS I and II, carry minimal operational consequence, as both classes represent patients appropriate for ambulatory and outpatient settings with equivalent resource requirements. As classification rank increases, the stakes escalate: a one-class error at the ASA-PS II-III boundary carries meaningful downstream consequences because many ambulatory surgery centers and office-based facilities use ASA-PS thresholds as explicit eligibility criteria, so an under-classified ASA-PS III patient may be routed to a facility unprepared for their acuity, while an over-classified patient may unnecessarily consume high-acuity resources. ASA-PS is, however, only 1 determinant of venue: procedural risk should codetermine where surgery occurs, and an ASA-PS III patient whose comorbidities require no additional clinical support may safely undergo a low-risk procedure such as cataract surgery in an ambulatory surgery center [30]. Preoperative evaluation and testing should likewise be matched to the surgical task at hand, an alignment identified as a primary source of net cost reduction in the perioperative surgical home model [31]. By effectively eliminating errors of 2 or more classes, the parallel ensemble prevents the most consequential misclassifications, those that would route high-acuity patients into fundamentally inadequate care environments.

This work builds upon and advances multiple approaches to automated ASA-PS classification. Prior efforts have explored 3 primary strategies: structured data-only models, text-only models, and other advanced approaches.

Text-based and retrieval-augmented approaches provide the closest comparators. Chung et al [16] evaluated multiple natural language processing models on preoperative notes from 19,740 patients, with fastText (Facebook AI Research [FAIR], Meta Platforms Inc.) achieving an accuracy of 0.617 and BioClinicalBERT 0.584; Yoon et al [32] ClinicalBigBird achieved an accuracy of 0.728, outperforming board-certified anesthesiologists in specificity, precision, and F₁, though within a single South Korean health system whose highly standardized documentation and uniform data structures likely facilitated stronger model performance; and Chen et al [11] pioneered a retrieval-augmented generation (RAG) approach, using GPT-4 with retrieved similar cases to achieve 74.09% accuracy. Our parallel ensemble improved upon Wongtangman’s structured-data approach [13] despite using a similar feature set, likely attributable to our integration of LLM-extracted diagnoses from unstructured clinical text.

Our parallel ensemble’s macro-F₁-score of 0.530 (95% CI 0.350‐0.600), exact match accuracy of 0.630 (95% CI 0.580‐0.670), and ±1-class accuracy of 0.980 (95% CI 0.968‐0.992) advance these efforts in 3 key ways. First, no prior team has used a parallel ML-LLM architecture for ASA-PS prediction. Second, while Chen et al [11] RAG approach achieved slightly higher accuracy (74.09% vs 63%), our model provides structured explanations without requiring a retrieval database, making it more deployable in resource-constrained settings. Finally, our performance metrics approach the theoretical ceiling (F₁-score=0.59) established by expert macro-F₁-score against the clinical reference standard while maintaining safety through near-perfect extreme error prevention, a metric not reported in prior studies but critical for clinical deployment. The consistent finding across all studies, including ours, is that neither structured nor unstructured data alone maximizes performance. Our contribution demonstrates that parallel architecture with transparent weighting offers the most promising path toward clinically deployable ASA-PS automation.

This study has limitations that warrant consideration. First, as with all research involving the ASA-PS system, the absence of a verified gold standard remains a fundamental challenge. Because ASA-PS assignment is inherently subjective, the model is trained to replicate human behavior rather than an absolute physiological truth. Consequently, perfect alignment between the model and provider is neither possible nor desirable. While the ideal ground truth would be a massive hand-labeled dataset derived from an expert panel of anesthesiologists, the thousands of encounters required for robust model training make such a dataset difficult and time-intensive to obtain.

Second, the LLM-generated explanations (Textbox 2) represent a proof-of-concept rather than production-ready outputs. While these explanations are not likely to satisfy every anesthesia provider, they demonstrate a critical principle: transparent algorithmic reasoning is essential for clinical adoption. Anesthesiologists will not trust black-box ASA-PS predictions; they require visibility into the model’s decision-making process to validate or challenge automated assessments. These preliminary outputs show that the model considers clinically relevant factors (comorbidities, functional status, and treatment burden) aligned with how providers conceptualize ASA-PS scoring. Though requiring substantial refinement before deployment, this transparency framework illustrates how AI could serve as a cognitive multiplier in resource-constrained preoperative clinics by helping triage thousands of patients, flagging complex cases for intensive review while streamlining straightforward evaluations. By making reasoning explicit, we establish the foundation of trust necessary to realize efficiency gains without compromising care quality.

Lastly, this study was conducted at a single institution with 3000 encounters in which ASA-PS V cases were extremely rare. Also, race and ethnicity data were not available and therefore could not be evaluated. Race and ethnicity fields are not captured within the structured-data pipeline used for this study, so the fairness audit was necessarily limited to language, gender, and age subgroups. Because race and ethnicity are well-documented determinants of documentation patterns and electronic health record data quality [33], their absence meaningfully constrains the comprehensiveness of the fairness evaluation, and prospective validation will prioritize their inclusion. A post hoc subgroup fairness audit on the held-out test set identified a substantial performance gap for Spanish-speaking patients (n=13; macro-F₁=0.216; accuracy=0.385), corresponding to deficits of 31.4 and 24.5 percentage points below the overall benchmark, respectively. This likely reflects the predominance of English-language clinical documentation in both the source data and the LLM training corpus. This deficit should be interpreted with caution: computing macro-F₁-score across 5 classes on 13 observations is mathematically unstable, and the figure reflects a directional signal warranting further investigation rather than a precise quantitative estimate. Performance for gender subgroups was comparable to the overall benchmark. Age-subgroup fairness analysis was constrained by missing data in a subset of encounters and should be interpreted with corresponding caution. These equity findings warrant careful consideration before deployment in multilingual or demographically diverse settings, and prospective multilingual validation is a priority for future work. Additionally, the structured data inputs rely in part on a proprietary LLM-based diagnosis extraction pipeline, which was internally validated at a sensitivity of 0.738 (95% CI 0.719‐0.752), precision of 0.845 (95% CI 0.832‐0.857), and specificity of 0.978 (95% CI 0.976‐0.980), and extraction errors may propagate to the XGBoost component. XGBoost output probabilities were not formally calibrated prior to applying the confidence gating threshold in the sequential hybrid architectures; formal calibration using out-of-fold training predictions is recommended before clinical deployment. Within the parallel ensemble specifically, final predictions average the uncalibrated XGBoost probability distribution with the GPT-4o output. Several factors limit the practical impact of this miscalibration: the GPT-4o component contributes a hard classification encoded as a one-hot vector rather than a competing probability distribution, the XGBoost stream carries only 30% of the weighted prediction (alpha=.30), and macro-F₁-score is stable across alpha=.20‐.45, indicating low sensitivity to the precise XGBoost probabilities. Formal calibration via isotonic regression or Platt scaling on out-of-fold training predictions nonetheless remains preferable and should precede prospective deployment. These factors, together with the single-center design, constrain the generalizability of our findings. The 0.04 macro-F₁-score delta between validation (0.574) and held-out test (0.530) performance reflects the expected consequence of selecting the ensemble weight on a finite validation set, inherent to any train, validation, and test design; alpha was fixed at 0.30 prior to test-set evaluation with no further tuning. Overlapping bootstrap CI across both estimates suggest this gap reflects sampling variance rather than substantial overfitting, though evaluation on larger independent cohorts is needed to confirm generalizability.

The exclusion of emergency cases constrains operational scope, though deliberately so: true emergencies lack the complete preoperative documentation both pipelines require, and the emergency modifier reflects urgency rather than the comorbidity burden the model predicts. In Kahneman [34] terms, emergency care demands fast thinking, and in such rapidly evolving clinical contexts an automated baseline classifier becomes irrelevant. The framework is intended for the deliberative, slow-thinking setting of elective and semielective pathways, where documentation is complete and consistent baseline classification has operational consequences. Future research involving multicenter datasets, a broader ASA-PS class distribution, and prospective external validation is necessary to establish the robustness of this parallel ensemble framework across diverse perioperative environments.

ASA-PS classification, despite its limitations, remains integral to perioperative workflows, and reducing classification variability through standardized algorithms addresses a persistent operational challenge without supplanting clinical judgment. Ferrari et al [12] found that anesthesiologists agreed with algorithmic ASA-PS predictions in 90.7% of cases, suggesting that providers will accept automated classification, but that acceptance is conditional on the transparency requirement described above: anesthesiologists will not trust black-box predictions, and it is precisely this need that the explanation component of our framework is designed to meet.

The potential impact is greatest given current financial and workforce headwinds. Persistent shortages of nurses [35], anesthesiologists [36], and perioperative personnel [37] have strained preoperative evaluation capacity while surgical volumes [38] continue rising alongside an aging population with greater medical complexity [39]. These converging pressures demand scalable solutions that maintain quality while expanding access.

We present a parallel ML-LLM ensemble built from a single center that achieved 63% (95% CI 0.580‐0.670) exact match accuracy and macro-F₁-score of 0.53 (95% CI 0.35‐0.60) on a true held-out test set for ASA-PS classification, with 98% (95% CI 0.968‐0.992)±1-class accuracy, approaching the theoretical performance ceiling established through independent review by 2 board-certified anesthesiologists (AF and GF). This architecture represents the first parallel hybrid approach to ASA-PS prediction, demonstrating that independent processing of structured and unstructured data before weighted integration constitutes a dual-input model rather than a purely ML or LLM approach. By providing transparent explanations alongside predictions, this framework represents not merely a technical advance but a practical response to the evolving demands of perioperative medicine, with the potential to improve patient safety, operational efficiency, and access to care across diverse health care settings.