We carried out an observational cohort study using EHR data in the OMOP CDM format (version 5.3) from 2 tertiary hospitals, the Seoul National University Bundang Hospital (SNUBH) for development and internal validation and Seoul National University Hospital (SNUH) for external validation. SNUBH and SNUH are tertiary general hospitals affiliated with universities in the Seoul metropolitan area and the capital city, respectively. They provide services for inpatients, outpatients, and ED patients. The EHR data from both hospitals encompass patient demographics, diagnoses, chief complaints, outpatient drug prescriptions, inpatient and ED drug administrations, operations, vital signs, laboratory test results, and in-hospital deaths [24]. Both hospitals use the same EHR system and have similar ETL processes for OMOP CDM. Out of a total of 961,672 admission episodes, 137,437 (14.3%) indexed hospitalizations with LoS ranging from 2 to 30 days between January 2016 and December 2020 were included in the study. The prediction timing was set within 30 days, aligning with standard practices in most studies [25]. We included individuals who were over 18 years old, had at least 1 inpatient visit occurrence, were hospitalized in or after 2016, had at least 1 occurrence of any medical condition, survived hospitalization, and had no unplanned admissions during the study period. The study cohort of patients is provided in Figure 1.

Each participating site (SNUBH and SNUH) obtained an institutional review board (IRB) exemption (IRB no X-2103-672-907, IRB no E-2305-020-1429). Informed consent was waived due to the use of deidentified data within a secure internal network and the retrospective nature of the study.

We used an open-source Feature Extraction R package (version 3.0.1; R Foundation for Statistical Computing) to extract patients’ clinical data before hospital admission from the OMOP CDM at SNUBH. Patient clinical characteristics, including condition occurrence, drug, observation, measurement, procedure, death, and visit occurrence tables were collected. The covariates included sex, age, the department at the visit, surgical operations, and lab results. After analyzing the LoS distribution for all patients, we excluded 2 departments, ophthalmology and nuclear medicine, from the model development due to their significantly shorter LoS (Figure S1 in Multimedia Appendix 1). Additional covariates that relied on the number of days (–365, –180, and –30) relative to the index date included diagnosis history, visit occurrences, and drug history. Severity scores such as the Charlson Comorbidity Index, Diabetes Complications Severity Index, CHADS2, and CHADS2-VAsc were also included as covariates. Additionally, custom covariates, such as the time, day of the week of admission, admission on holidays, day of the week of operation, whether operations were performed, diastolic and systolic blood pressure, body temperature, diagnosis at admission, and anthropometric measurements (eg, height and weight) were incorporated in the development of the prediction models.

The study classified hospitalizations lasting 7 days or longer using binary classification. In comparison, multiclassification was applied to categorize hospital stays lasting 3 days or longer, with specific subgroups for 3, 4, 5, 6, and 7 days. A hospitalization was defined as an inpatient visit during the study period, excluding unplanned admissions such as hospitalization through the emergency room [26].

After generating all the custom and noncustom covariates, logistic regression (LR) with Lasso regularization eliminated redundant collinear predictors from the models. We normalized continuous variables for uniform feature representation during data preprocessing and converted categorical variables into binary vectors using one-hot encoding. We removed variables from the analysis that had over 30% of the values missing [27]. Six ML models [5,28], including LR, random forest (RF), Extreme Gradient Boosting (XGB), Light Gradient Boosting (LGB), Gradient Boosting (GB), and multilayer perceptron (MLP) were evaluated to develop predictions for binary (LoS ≥7 days) and multiclass length of stay (LoS 3, 4, 5, 6 ≥7 days) outcomes based on the selected features. Model hyperparameters were optimized using 10-fold cross-validation via RandomizedSearchCV from Python (v.3.7.6) and scikit-learn library (v.0.22.1). We externally validated the models using SNUH data.

We evaluated model performance using 5 metrics: area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), sensitivity, precision, and specificity. A bootstrap resampling technique with 1000 replicates was employed to compute 95% CIs for all metrics. The optimal threshold point of the Youden Index was used to assess the sensitivity and specificity of an LoS of 7 days or longer. Shapley Additive Explanations (SHAP) analysis interpreted the output of ML models and identified predictors that positively and negatively contribute to the model’s prediction. Furthermore, calibration plots assessed the reliability of the prediction models and ascertained whether the predicted probabilities aligned with the actual outcomes across different probabilities. The Brier Score measured the accuracy of probabilistic predictions quantitatively. The performance of the models at SNUH was evaluated using AUROC and AUPRC, SHAP, and calibration plots alongside Brier Scores.

Statistical analysis was conducted using R software (version 3.0.1) and Python (version 3.8.10).

Patient entry events consisted of 2 groups based on LoS <7 days and LoS ≥7 days. For the internal data set, comparisons between these groups were performed using standard 2-sample t tests for continuous variables and chi-squared tests of independence for categorical variables. To compare characteristics between internal and external validation cohorts, we used the tableone package in R, which generated descriptive statistics presenting continuous variables as mean (SD) and categorical variables as frequency counts and percentages. All the packages used are listed in Table S2 in Multimedia Appendix 1.

During the study period, we identified 129,938 inpatient admission episodes at SNUBH and 169,165 at SNUH, each with an LoS ranging from 2 to 30 days. The mean age of the admitted patients were 58.4 (SD 15.4) years at SNUBH and 59.2 (SD 14.9) years at SNUH. The average LoS for planned admissions was 5.2 days at SNUBH and 5.4 days at SNUH. The baseline characteristics of both cohorts are presented in Table 1.

In the internal validation of the prediction model, the XGB model achieved the top performance in planned admissions for binary classification, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) (Table 2). The LGB model showed the highest performance in the multiclassification, with an AUROC of 0.901 (95% CI 0.898-0.904), 0.836 (95% CI 0.831-0.841), 0.750 (95% CI 0.742-0.758), 0.766 (95% CI 0.757-0.774), and 0.890 (95% CI 0.887-0.893) for LoS 3, 4, 5, 6, and ≥ 7 days, respectively, and a macro AUPRC of 0.556 (Table 3). Overall, the predictive performance for LoS consistently showed good discriminatory ability, achieving an AUROC >0.800 in both the binary and multiclassification scenarios (Figure 2A, Table 2).

The most important features contributing to the predictive performance of the models were the operation performed, the admitted patient’s department, age, and severity scores such as the Charlson Index-Romano adaptation and the frequency of outpatient visits in the past 6 months (Figure 3A). In the multiclassification models, the SHAP analysis revealed distinct predictors, including surgical operations, blood pressure, and day of the week of hospital admission, which significantly influenced different hospital stay durations (Figure S2 in Multimedia Appendix 1). Specifically, surgical operations were significant predictors for LoS of 3 days and 7 days or longer, while admissions in obstetrics/gynecology and neurosurgery were relevant for LoS of 4 and 5 days. Additionally, hospital admissions on Wednesdays and Fridays were associated with a 6-day LoS (Figure S2 in Multimedia Appendix 1). The calibration plot and Brier scores indicated strong calibration performance, achieving approximately 0.14 scores in both the binary and multiclassification models (Figure 4A, Table S3 in Multimedia Appendix 1).

In the external validation, the RF model performed the best in predicting an LoS of 7 days or longer with an AUROC of 0.804 (95% CI 0.802-0.807) and an AUPRC of 0.545 (95% CI 0.540-0.551) (Table 3). The AUROC decreased by 8% compared to the AUROC obtained from SNUBH (Figure 2B). The LGB model showed superior overall performance for multiclassification, with an average AUROC of 0.711 and a macro AUPRC of 0.383 across different LoS durations (Table 3). The AUROC showed a 0.93 reduction compared to the internal validation multiclassification model.

The SHAP analysis identified significant predictors for hospital stays of 7 days or longer, including surgical procedures, elevated body temperatures, frequency of outpatient visits in the past month, hospital admissions on Fridays, and older age (Figure 3B). In the multiclassification analysis, akin to the prediction model’s internal validation, cases with surgery performed showed an increased LoS in the hospital. Admission to the cardiology department generally led to discharge within 3 days. Specifically, admissions to the obstetrics/gynecology and hematology/oncology departments were associated with an average LoS of 4 and 5 days, respectively. Hospital admissions on Sundays and a higher number of outpatient visits in the past month resulted in hospital stays lasting more than 6 days (Figure S3 in Multimedia Appendix 1). The binary and multiclassification models accurately calibrated predicted probabilities and actual outcomes on the external validation data set (Figure 4B, Table S4 in Multimedia Appendix 1).

In this paper, we proposed two ML models using the standardized vocabularies of the OMOP CDM: (1) a binary classification model predicting LoS of 7 days or more and (2) a multiclassification model categorizing LoS as 3, 4, 5, 6, and 7 days or longer. This study demonstrates that combining clinical and nonclinical characteristics from a standardized data source from different institutions yields moderate predictive accuracy and efficiently leverages the benefits of multicenter studies. To our knowledge, our model is the first to offer both binary and multiclass classification predictions for LoS using the OMOP CDM.

The ML algorithms using GB performed the highest binary classification performance, while the LGB model performed the best in the multiclass classification. The ML predictions identified surgical operation, diastolic blood pressure, and the day of week of hospital admissions as critical factors for predicting both an LoS of 7 days or more and for multiclass classification of LoS. The external validation revealed a model performance 8% lower than the internally calculated AUROC value. This decline in performance could be attributed to variations in patient populations and medical specialties across hospitals and disparities in converting EHR data into the standardized OMOP CDM format [29-31].

In this study, we conducted an extensive analysis across 46 departments, developing a model that predicted LoS from 3 to 7 days or longer. A multidepartment approach unraveled the complex dynamics of patient stays and illuminated the varied factors influencing each day’s length of stay. Insights from this broad patient spectrum could equip health care professionals with essential tools for improving patient care strategies in various health care settings. These models may aid in optimizing health care resources and contribute to identifying patients most likely to be discharged on a particular day using classification algorithms.

Additionally, our LOS prediction model offers potential economic benefits by enabling better planning and management of bed occupancy, reducing costs associated with the inefficient use of hospital beds. By anticipating extended stays, hospitals can arrange necessary interventions in advance, potentially preventing prolonged hospitalizations and lowering treatment costs. Streamlining discharge processes based on accurate predictions reduces delays and frees up beds more quickly, leading to cost savings in terms of staffing and resource management. Although further studies are needed to empirically validate these benefits, our findings suggest that implementing such predictive models could support clinical and financial improvements in hospital management.

Earlier studies often used data sources such as administrative data, the MIMIC, and EHRs for model development, resulting in limited applicability across different institutions due to heterogeneous data standards (Table S1 in Multimedia Appendix 1). The variations in these data sources hindered scalability and the capacity to efficiently manage a wide range of routinely collected observational health data. Previously utilized data constrained the transferability of the model outputs to different patient populations across multiple medical specialties and institutions [5,17,32]. We designed prediction models that implemented the OMOP CDM standardized vocabularies and demonstrated their application across independent institutions, ensuring interoperability and applicability. Although one study predicted LoS from multicenter data using federated learning, it relied solely on administrative data and trained with 3 regression models, suggesting multicollinearity issues [33]. Unlike the aforementioned model, ours included hospital stay characteristics and patients’ medications incorporating more records than the federated learning research. As a result, our model, including hospital admission information and comprehensive patient history, provides more reliable insights, covering a broader range of records.

Systematic reviews assessing efficacious health system interventions to avoid prolonged hospital LoS in high-risk populations indicate that reducing LoS requires a multifaceted approach, including clinical care and the logistics of care coordination [4]. However, most previous studies primarily used demographics, lab results, and vital signs as input features [1]. Some authors integrated data from different modalities, combining free-text notes and demographic information with time-varying features, while only a few employed medications as predictive variables [1]. In this study, we incorporated clinical and nonclinical features, including medical specialties, severity scores, day and time of admission, and hospital admissions during the weekend. Time-dependent covariates classified as occurring in the past 1, 6, and 12 months include frequency of different types of visits, active ingredients in a drug group, and medical conditions to capture health care utilization patterns. This comprehensive data integration provides a longitudinal view of the patient, enabling more personalized care planning and effective discharge strategies tailored to each patient’s recovery.

When comparing prediction models for a 7-day LoS using the OMOP CDM within the 2 tertiary hospitals, we found that surgical operation, admitting unit, patient age, severity scores, and admission day significantly influenced hospital stay duration. For example, our internal validation revealed that older patients admitted to the neurosurgery department and undergoing surgery often have longer LoS, aligning with the notion that older adult patients requiring neurosurgery tend to have poorer prognosis and generally require extended hospitalization [34]. In contrast, admissions to obstetrics/gynecology appeared to have a shorter LoS, likely because they are mostly elective surgeries, such as cesarean deliveries [35,36]. A lower frequency of outpatient visits in the past 6 months was associated with a longer LoS, indicating that patients who visit the hospital less frequently may not be managing their health effectively. Friday admissions, typically through the emergency room, were more likely to involve severe conditions, while the unavailability of elective surgeries during the weekend could result in a longer LoS [37]. Weekend admissions, particularly on Sundays and Mondays, were associated with shorter LoS, likely due to the scheduling of preplanned treatments and elective surgeries on weekdays [38]. Additionally, certain medical specialties were linked to extended LoS. Psychiatric admission often resulted in longer stays, attributed to the need for comprehensive assessments and stabilization in mental health care [39]. Similarly, admissions to hematology/oncology departments frequently reported prolonged stays for monitoring and managing side effects and complications arising from hematologic cancer treatments [40]. Higher body temperatures indicated more severe conditions, potentially leading to longer durations of illness [41].

This study has several limitations. The models were only externally validated in a single hospital, which may limit their applicability to other health care settings with different patient populations and characteristics. However, health care systems with OMOP CDM can readily implement the methods proposed in this study using standardized vocabularies and analytic codes [17,23,42]. This work aids in developing a model that predicts the remaining LoS based on hospitalization progression during the admission period within a heterogeneous population through the Observational Health Data Sciences and Informatics (OHDSI) network study. Additionally, smaller patient numbers in the 5- and 6-day LoS categories in both the internal and external validation sets (Table S5 in Multimedia Appendix 1) may have affected the model performance for these subgroups. The models may not fully capture real-world performance due to the limited consideration of local context and resources such as team dynamics, hospital processes, staffing resources, social drivers of health, and administrative support—limitations stemming from nonstandard fields in the OMOP CDM. Using the standardized information we could capture from the CDM, we developed a reproducible model based on standardized variables that are less influenced by individual hospitals, such as clinical factors and hospital stay characteristics.

In this study, we utilized the OMOP CDM to predict the LoS in planned admissions, considering both binary (LoS ≥ 7 days) and multiclassification scenarios (LoS of 3, 4, 5, 6, and 7 days or more). The performance of the prediction models was consistent and confirmed through external validation. The proposed models showed ease of application in various clinical settings and reproducibility across institutions within the OHDSI community. The findings of this study may assist hospitals in effectively managing hospital resources, such as staffing, equipment, and supplies at different institutions.