This was a multicenter, noninterventional, retrospective study using data captured in the EHRs of the participating hospitals within the Servicio de Salud de Castilla-La Mancha (SESCAM) Health Care Network in Castilla-La Mancha, Spain (Figure 1). Data captured in EHRs were collected from all available departments, including inpatient hospital, outpatient hospital, and emergency department (ED), for virtually all types of provided services in each participating hospital. The study period was January 1 to March 29, 2020.

The study database was fully anonymized in a structured format and contained no personal information from patients. Likewise, personal information was not accessed during either the application of automated and algorithmic methods (ie, NLP) or the conversion of unstructured data into the structured database. Importantly, given that clinical information was handled in an aggregate, anonymized, and irreversibly dissociated manner, patient consent regulations do not apply to the present study.

The study sample included all patients in the source population who were diagnosed with COVID-19. Patients were identified on the basis of clinical diagnosis or microbiological test results. Clinical confirmation of COVID-19 cases was determined by observed symptomatology, imaging (mostly chest X-ray), and laboratory results, as captured in the unstructured, free-text information in the EHRs. Microbiological test result confirmation of COVID-19 cases involved reverse transcriptase–polymerase chain reaction (RT-PCR) or similar available tests. Our decision to consider cases confirmed both clinically and by RT-PCR was justified by the limited availability of routinely administered RT-PCR tests in the region during the study period and supported by recent discussions on the far-from-optimal sensitivity of RT-PCR for COVID-19 (ie, a single negative result from a single specimen cannot exclude the disease in suspected cases) [20,21]. Indeed, recent reports highlight the clinical validity and relatively high sensitivity of symptom- and imaging-based identification of patients with COVID-19, especially in early stages of the disease [20,22,23].

To meet the study objectives, we used EHRead [24], a technology developed by Savana that applies NLP, machine learning, and deep learning to analyze the unstructured free-text information written in millions of deidentified EHRs. This technology enables the extraction of information from all types of EHRs and subsequent normalization of the extracted clinical entities to a unique terminology. This process enables further analysis of a descriptive or predictive nature. Originally based on Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) terminology, our unique body of terminology comprises more than 400,000 medical concepts, acronyms, and laboratory parameters aggregated over the course of five years of free-text mining, targeting the most common diseases (eg, respiratory diseases, cardiovascular diseases, and diabetes).

Using a combination of regular expression rules and machine learning models, the terminology entities are detected in the unstructured text and later classified based on sections typically contained in EHRs, hospital services, and other clinical specifications. Importantly, each detected term is described in terms of negative, speculative, or affirmative clinical statements; this is achieved by using deep learning convolutional neural network classification methods that rely on word embeddings and context information (for a similar methodological approach, see [25]). The limitations of case-by-case detection are also overcome with a similar approach to ensure that the detected concepts are used within the appropriate context for the descriptive and predictive analysis.

For particular cases where extra specifications are required (ie, to differentiate COVID-19 cases from other mentions of the term related to fear of the disease or to potential contact), the detection output was manually reviewed in more than 5000 reports to avoid any possible ambiguity associated with free-text reporting. All NLP deep learning models used in this study were validated using the standard training/validation/testing approach; we used a 75/12/13 split ratio in the available annotated data (between 2000 and 3000 records, depending on the model) to ensure efficient generalization on unseen cases. For all developed models, we obtained F scores greater than 0.89.

All categorical variables (eg, comorbidities, symptoms) are shown in frequency tables, whereas continuous variables (eg, age) are described via summary tables that include the mean, SD, median, minimum and maximum, and quartiles of each variable. To test for possible statistically significant differences in the distribution of categorical variables between study groups (ie, male vs female, ICU admission vs no ICU admission), we used Yates-corrected chi-square tests. For continuous variables, mean differences were tested using t tests. Given our general population approach and our unusually large sample size, we were interested in exploring sex-related differences in patients with COVID-19; therefore, most results were stratified by sex [26]. All statistical inferences were performed at the 5% significance level using 2-sided tests or 2-sided CIs.

We developed a decision tree to classify patients with COVID-19 according to their risk of being admitted to the ICU. The two types of patients or classes considered in the model were therefore “admitted to the ICU” and “not admitted to the ICU.” The model maps the characteristics of the patients (the variables) to their class in the shape of a tree. From a clinical perspective, this model contemplates all patient variables upon admission; therefore, it is predictive from symptom debut until outcome. The tree is composed of nodes that branch to subsequent child nodes depending on the patient’s variables. The tree is built in such a way that each branch separates the two classes as much as possible. This separation is measured as the Shannon entropy, where a node with an entropy of zero indicates that the classification is perfect (either all or none of the patients were admitted to the ICU) and an entropy of one is the worst possible mix (50%/50%) [27].

The model was developed and tested on the available data from hospitalized patients who had or had not been admitted to the ICU; the latter were either discharged from the hospital or died in the course of the disease. This amounted to a total of 900 patients. We validated our algorithm by splitting our COVID-19 sample into a 70% training set and a 30% validation set. This means that the model was trained with 630 patients (582 who did not require intensive care vs 48 who did) and validated over the remaining 270 patients. Because the two classes were unbalanced (far fewer patients required ICU admission), we used the standard technique of oversampling the lower class to guarantee a balance of accuracy and recall (ie, the tradeoff between false positives vs false negatives). Further, we sought to replicate the results of this validation in an a posteriori sensitivity analysis, as per recent recommendations for predictive modeling in COVID-19 [28] and TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) guidance [29]. For this second validation, we trained the model with data from the provinces of Ciudad Real and Guadalajara (38% of the study sample from Castilla-La Mancha), and we used an independent set with combined data set from the other three provinces, namely Toledo, Cuenca, and Albacete, for validation.

Additional details regarding the development and validation of the predictive algorithm are included in the Supplementary Methods in Multimedia Appendix 1. The workflow used for the generation of the predictive algorithm is summarized in Figure S1 in Multimedia Appendix 1.

By accessing the clinical information of more than 10,000 anonymous patients with COVID-19 (a number that largely surpasses samples included in recent reports about the disease [30,31]), we were able to describe their demographic and clinical characteristics, their clinical journey, and the statistical relationship between the most common symptoms and comorbidities on admission, and COVID-19 prognosis (ie, ICU admission). There were subtle differences in clinical symptoms at onset by sex, while all comorbidities (except asthma) were significantly more frequent in male than female patients with COVID-19. The variables identified in our ICU admission model (ie, age, temperature, and tachypnea) are clinically relevant, as they are readily available and easily observable in everyday practice for patients with COVID-19. Although tachypnea is not an exclusive manifestation of COVID-19 and can be present in patients suffering from other respiratory diseases (ie, pneumonia), our model suggests that this sign (in combination with age and temperature) is a more reliable predictor of ICU use than other common symptoms and signs, such as cough, dyspnea, or respiratory crackles.

The reported sex-dependent differences in the symptomatology of COVID-19 at onset have been further confirmed by our group using similar methods [32] and should be interpreted in light of data suggesting that female teenagers and young adult women are significantly more affected by the disease than their male counterparts [32]. In this regard, our results warrant further investigations aimed at closing the gender gap in the ongoing pandemic [33].

Given that the stability and capacity of ICUs worldwide is threatened by the rapid spread of COVID-19, the identification of individual factors that predict ICU admission may not only improve patient management but also optimize health care resource use and planning. Thus, recent studies using big data and machine learning have explored the prognostic factors of the disease, including ICU transfer, discharge, and mortality [8-11]. In line with our results, respiratory rate has also been identified as an important predictor of ICU transfer in patients with COVID-19 [9].

If further applied to other national and international health care networks, the tools and methodology presented in this study can potentially characterize and predict the prognosis of COVID-19 in a timely and unprecedented manner. As demonstrated in recent studies [34,35], there may be value in the application of AI to the current COVID-19 pandemic, not only to predict outbreaks [36] or read chest computed tomography scans [37] but also to elucidate the clinical onset and natural history of COVID-19 almost in real time. Indeed, classical methods would require months of questionnaire-based data collection and questionnaire validation along with multiple Ethics Board approvals and other practical hurdles; these steps can be avoided with our current approach.

In the race against COVID-19 [38], where the goal is to curb the pandemic, it is imperative to leverage big data and intelligent analytics for the betterment of public health. However, it is of the utmost importance not to neglect privacy and public trust, to apply best practices, and to maintain responsible standards for data collection and data processing on a global scale [39].

To our knowledge, this is one of the first attempts to combine NLP and machine learning to access and analyze unstructured, free-text real-world data from EHRs in a large series of patients with COVID-19. Although recent studies have used machine learning to predict ICU admission in patients with COVID-19 [9], our approach takes this methodology one step further by applying NLP to exclusively analyze unstructured information. Indeed, our state-ot-the-art methodology enabled rapid analysis of the unstructured free-text narratives in the EHRs of 1 million patients from the general population of the region of Castilla-La Mancha (Spain).

Our methodology combined modules for sentence segmentation, tokenization, text normalization, acronym disambiguation, negation detection, and a multidimensional ranking scheme; the latter involved linguistic knowledge, statistical evidence, and continuous vector representations of words and documents learned via shallow neural networks. When applied to EHRs, NLP enables both access to longitudinal health records for all patients in the target population and the implementation of exploratory analysis to clarify associations between variables that have remained undetected with traditional research methods. By considering all possible patients with the target disease, the information and analyses used here (ie, real-world data and free-scale statistics) remained unbiased by the research question or the observers. Unlike classical statistical methods (eg, logistic regression), the main advantage associated with the use of machine learning in this context is that it enables the automatic detection of meaningful relationships between variables. For instance, if a given symptom (ie, fever) is only relevant for certain patients (ie, older than 50 years), techniques such as the classification trees used here are suitable to uncover this relationship. In this context, although the total number of patients who required ICU use in the training set was somewhat low (48 patients), the number of variables considered in the model was also very limited. In addition, the inclusion of a validation stage reduces the likelihood of overfitting. Ultimately, the use of classification trees in this study (as opposed to other models, such as artificial neural networks) is especially appropriate in the clinical context because they are easily interpretable.

Regarding the geographical location of our participating hospital sites, it is worth mentioning that with a total of 1,364,924 patients from the region of Castilla-La Mancha (SESCAM Health Care Network), our sample is representative of the Spanish population. Spain is among the countries that have been most affected by the pandemic in terms of both total cases and mortality rates [40,41]; the Castilla-La Mancha region in particular is the third most affected region in the country, just behind Madrid and Catalonia. For this reason, we anticipate that the clinical conclusions drawn here will be relevant for clinicians worldwide. Of note, the ICU capacity in the region during the study period had not yet been compromised, which protects against possible bias in our training data (all patients requiring intensive care were indeed admitted to the ICU).

The results and conclusions of the present study should be interpreted in light of the following limitations. First, we did not distinguish COVID-19 cases confirmed by laboratory results (ie, RT-PCR) from those exclusively diagnosed through clinical observation (ie, symptomatology, imaging and laboratory results). However, it should be noted that PCR and other rapid laboratory tests for the detection of SARS-CoV-2 were not routinely administered in Spain during the study period. In addition, this decision is supported by recent discussions on the clinical validity and relatively high sensitivity of symptom- and imaging-based identification of patients with COVID-19, especially in early stages of the disease [20,22,23]. Second, independent replications by different research groups in larger patient sets are needed to further support the clinical validity of our results.

Finally, future reports from the BigCOVIData study may incorporate laboratory results and treatments and may contextualize the results presented here in a larger clinical picture [28].

In this study, we found that in the largest series of patients with COVID-19 attended during the first three months of the pandemic in Spain, 6.1% of all hospitalized patients (83/1353) required ICU admission. We also found that a combination of easily obtained clinical variables, namely age, fever, and tachypnea, predicts whether patients with COVID-19 will require ICU admission.