Health care organizations (HCOs) change intensive care unit (ICU) staffing and follow physical distancing policy during the COVID-19 pandemic to protect clinicians and patients [1,2]. For instance, many physical care activities before the COVID-19 pandemic have been transitioning to virtual systems, such as electronic health records (EHRs) or telehealth [3-5]. These changes can interfere with the structures of teamwork in the ICU, which may impact clinical outcomes. The changes in ICU structures and outcomes from pre– to intra–COVID-19 periods have not been systematically investigated. Therefore, challenges are unclear when health care delivery disruptions (eg, pandemics) or major transitions (physical to virtual collaboration) occur in the post–COVID-19 era.

One of the major challenges to analyzing ICU structures and quantifying their changes is that the ICU structures are historically developed at a coarse-grained level, which seldom considers connections among clinicians in a team owing to dynamic and complex clinical workflows, shifts, and handovers [6-9]. Understanding how clinicians connect (eg, sharing and exchanging health information) within their clinical teams when caring for patients can provide fine-grained evidence to potentially refine or optimize existing ICU structures.

In modern health care, an increasing number of clinicians utilize EHR to diagnose and treat patients by exchanging all medical statuses [10,11]. Therefore, the volume of the EHR system utilization data has been increasing exponentially in recent years, providing abundant resources to identify connections between clinicians. Recent studies applied network analysis to EHR utilization data to measure connections among clinicians [12-15]. They found EHR system utilization data to potentially be a rich resource to be leveraged to model relationships among clinicians. Recent studies have also shown that network analysis methods and data within the EHR can also be utilized to learn collaboration structures in ICUs [8,9,16]. Based on previous studies, this study leverages network analysis methods to learn structures of the neonatal ICU (NICU) in pre– and intra–COVID-19 eras in terms of collaboration among clinicians, and compares differences in the structures. Patients hospitalized in the NICU include high-risk infants who may be or are at risk for a variety of complex diseases or conditions. The management of NICU patients typically requires multidisciplinary care teams (eg, neonatal frontline providers, ancillary staff, nurses, neonatologists, residents, support staff, respiratory therapists, neonatal fellows, and highly specialized consultants) [17-19]. We investigate the connections among clinicians in a tertiary-level NICU, which has a high density of intense EHR utilization and heavy data sharing traffic per patient episode [8,9], making this environment ideal for our ICU structure study.

To describe our work systematically, we used the reporting checklist for quality improvement in health care (Multimedia Appendix 1), which is based on the SQUIRE 2.0 guidelines [20].

We extracted EHRs for all patients admitted to the NICU at Vanderbilt University Medical Center (VUMC, Nashville, Tennessee) between September 1, 2019, and June 30, 2020. We characterized pre–COVID-19 as the period of September through December 2019 and intra–COVID-19 as the period of March through June 2020. We used network analysis methods to analyze the EHRs of 712 NICU patients (pre–COVID-19 patients, n=386; intra–COVID-19 patients, n=326), excluding those with COVID-19, to assess clinician networks to describe pre–COVID-19 and intra–COVID-19 teamwork structures. These 2 groups were compared using patients’ clinical characteristics, including their age, sex, race, length of stay (LOS), and discharge dispositions.

To protect patient confidentiality, analysis of EHR data was conducted at a data analysis server located at the VUMC. The EHR data used in this study were physically housed in a secure room at the VUMC’s data center. All connections made to the servers were made in an encrypted manner and used Secure Shell technology from known computers. A unique login and password were set for each authorized individual. All protected health data remained on the server, and no copy of the data was provided to unauthorized parties. The Vanderbilt institutional review board reviewed and approved the study (protocol No. 200792).

Clinicians’ EHR actions stemmed from different tasks, including conditions (eg, assessing a patient’s condition), procedures (eg, intubation), medications (eg, prescription drugs), notes (eg, progress note writing), orders (eg, laboratory test ordering), and measurements (eg, measuring blood pressure). We leveraged the actions committed to the EHRs of patients by clinicians to measure clinician-clinician connections. Prior research shows that a 1-day window can capture meaningful collaborative relationships among clinicians [12-15]. Based on their findings, we characterized a collaboration relationship (tie) between 2 clinicians as they performed actions to EHRs of the same patient within the same day (24 hours). This definition can capture different types of interactions between clinicians. The first type is the asynchronous interactions between clinicians; for instance, a clinician created a medication order at 9 AM, and another clinician reviewed and processed the order at 11 AM on the same day. Thus, the 2 clinicians had an asynchronous collaboration in terms of order creation and processing. The second type is the interactions between clinicians during shifts or handoffs, which are among the most critical aspects of collaboration owing to the medical errors that occur during the transition between clinicians [17-19,21]. For instance, 2 nurses (oncoming and ongoing) were responsible for a patient’s handoff or shift at 7 PM. The 2 nurses may perform some actions to EHRs of the patient before, during, or after the handoff/shift, which can be captured by our definition of collaboration relationship, to build a connection between them. The third type is the interactions between clinicians built on their documentation in EHRs or messages in the basket, a communication hub where clinicians can send and receive secure messages. For instance, 2 clinicians work together during patient care without interacting with EHRs, but both made some documentation later. Based on our collaboration definition, the 2 clinicians still have a connection built between them. We quantified the weight of a relation between 2 clinicians as the number of patients they co-managed on the same day, which can be learned from EHRs. We referred to each patient on that day as a patient day. The relation’s final weight is the cumulative number of patient days the 2 clinicians interacted by co-managing patients during our investigated time window (4 months). Thus, we built the pre– and intra–COVID-19 networks.

Formally, the nodes in the pre– and intra–COVID-19 networks were defined as Z_pre = {z₁, z₂, …, z_p} and Z_intra = {z₁, z₂, …, z_q}, respectively. To better interpret the networks, we used the clinician’s specialty (eg, respiratory therapist or NICU registered nurse) to label each node. Specialties in the pre– and intra–COVID-19 networks are referenced as EXP_pre = {exp₁, exp₂, …, exp_a} and EXP_intra = {exp₁, exp₂, …, exp_b}, respectively. Z and EXP are used to describe the compositions of the pre– and intra–COVID-19 networks.

We leveraged sociometric measurements, including eigenvector centrality [22], betweenness centrality [23], and eccentricity [24], to quantify the network structures. Eigenvector centrality is the measure of the influence of a node in a network [22]. In a health care EHR setting, high eigenvector centrality implies that the clinician has a very active role and serves as a hub in information sharing and dissemination. Betweenness is a measure of centrality based on the shortest paths in a network; it is calculated as the number of times a node acts as a bridge along the shortest path between two other nodes [23]. A node with higher betweenness centrality has more control in the network because owing to its shorter paths to other nodes, more information will pass through that node. Eccentricity of a node in a network is the maximum distance from the node to any other node [24]. In a clinician setting, eccentricity is the radius from one clinician to another, which is the largest distance. A larger eccentricity implies that there many more steps to share information with another clinician. Therefore, we characterized eigenvector centrality as indicating a clinician’s leadership (hub) in terms of collaboration, betweenness centrality as demonstrating a clinician cares for a wide spectrum (bridge) of patients, and eccentricity as showing the difficulty for a clinician to collaborate with others. Figure 1 shows 3 networks to illustrate each of the 3 sociometric measurements, respectively. We used Gephi, an open-source network analysis and visualization software package [25], to calculate eigenvector centrality, betweenness centrality, and eccentricity for each of the nodes in the pre– and intra–COVID-19 networks.

We investigated whether the differences in the clinician leadership, who care for a wide spectrum of patients, and collaboration difficulty are statistically different between pre– and intra–COVID-19 networks. In addition, we investigated changes in 2 outcome metrics, including LOS and discharge dispositions, from the pre– to the intra–COVID-19 periods. We applied Mann-Whitney U tests at a 95% CI to account for the non-Gaussian distribution of the sociometric measurements and outcomes. Since the pre– and intra–COVID-19 networks are composed of clinicians with different specialties, we compared the differences at both network (entire network) and specialty (each specialty) levels. We applied a Bonferroni correction to account for multiple hypothesis testing (eg, pairwise test for the specialty-level comparisons).

For each investigated specialty (eg, NICU nurse), we investigated whether clinicians affiliated with the specialty in the pre–COVID-19 network have significantly higher values of eigenvector centrality, betweenness centrality, or eccentricity than those affiliated with the same specialty in the intra–COVID-19 network. We focused on the 6 specialties that play essential roles in NICU care, which included NICU nurses, nurse practitioners, residents, respiratory therapists, cardiac ICU nurses, and neonatologists. For an investigated specialty, we created 2 arrays, one with values of a sociometric measurement (eg, eigenvector centrality) of nodes (clinicians) affiliated with the specialty in the pre–COVID-19 network and the other for the values of the same sociometric measurement of clinicians affiliated with the same specialty in the intra–COVID-19 network.

We further tested the differences in the eigenvector centrality, betweenness centrality, and eccentricity between pre– and intra–COVID-19 networks using all specialties in the 2 networks. The hypothesis test is of the following form: there are significant differences in eigenvector centrality, betweenness centrality, or eccentricity between the pre– and intra–COVID-19 networks. Within a network, we measured eigenvector centrality, betweenness centrality, or eccentricity for a specialty by calculating the mean values of eigenvector centrality, betweenness centrality, or eccentricity of all clinicians affiliated with that specialty in the network. For pre– or intra–COVID-19 networks, we developed an array of specialties, whose cell value is the value of a specialty’s sociometric.

The data sets generated and analyzed during the current study are not publicly available in order to maintain the privacy of the patient information investigated in this study, but they are available from the corresponding author on reasonable request.

The pre– and intra–COVID-19 patient groups share similar distributions (Table 1) in sex (~0 difference), race (4% difference in White, and 3% difference in African American), LOS (IQR difference in 1.5 days), and discharge dispositions (~0 difference in home, 2% difference in expired, and 2% difference in others). There were no significant differences in patient demographics and outcomes between the 2 groups.

There are several notable findings in network analysis to highlight. First, the intra–COVID-19 NICU structure had a higher eccentricity (collaboration difficulty) than the pre–COVID-19 (median 3 vs 4; P<.001). Second, NICU nurses had a lower eigenvector centrality (leadership in collaboration) in the intra–COVID-19 structure than the pre–COVID-19 (median 0.183 vs 0.087; P<.001). Third, neonatology physicians had a higher betweenness centrality (care for a wider spectrum of patients) in the intra–COVID-19 than in the pre–COVID-19 structure (median 0.0001 vs 0.005; P<.005).

Figure 2 shows the pre– and intra–COVID-19 networks from different perspectives. Figure 2A shows that the intra–COVID-19 network is larger, implying that the 6 types of clinicians are likely to be more difficult to collaborate with others (higher eccentricity) in the intra–COVID-19 NICU care than those in the pre–COVID-19 NICU care (median 4 vs 3; P<.001). This higher eccentricity in intra–COVID-19 implies that there is more distance from one node to any other node within the network. Compared to nurse practitioners and neonatology physicians, nurses’ leadership (eigenvector centrality) reduced from the pre– to intra–COVID-19 networks (median 0.183 vs 0.087; P<.001), as shown in Figure 2B. This reduced leadership implies that nurses were less active during the intra–COVID-19 period. Nurses do not occupy central positions in the intra–COVID-19 network, and they have more connections in the pre–COVID-19 period than in the intra–COVID-19 period, as shown in Figure 2C. Neonatology physicians care for a wider spectrum of patients (high betweenness centrality) in intra–COVID-19 NICU care than those in pre–COVID-19 NICU care (median 0.005 vs 0.0001; P<.001), as shown in Figure 2D. Therefore, during the COVID-19 pandemic, neonatology physicians have a higher number of shorter paths to other nodes within the network.