This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Telemedicine has been deployed by health care systems in response to the COVID-19 pandemic to enable health care workers to provide remote care for both outpatients and inpatients. Although it is reasonable to suspect telemedicine visits limit unnecessary personal contact and thus decrease the risk of infection transmission, the impact of the use of such technology on clinician workflows in the emergency department is unknown.
This study aimed to use a real-time locating system (RTLS) to evaluate the impact of a new telemedicine platform, which permitted clinicians located outside patient rooms to interact with patients who were under isolation precautions in the emergency department, on in-person interaction between health care workers and patients.
A pre-post analysis was conducted using a badge-based RTLS platform to collect movement data including entrances and duration of stay within patient rooms of the emergency department for nursing and physician staff. Movement data was captured between March 2, 2020, the date of the first patient screened for COVID-19 in the emergency department, and April 20, 2020. A new telemedicine platform was deployed on March 29, 2020. The number of entrances and duration of in-person interactions per patient encounter, adjusted for patient length of stay, were obtained for pre- and postimplementation phases and compared with
There were 15,741 RTLS events linked to 2662 encounters for patients screened for COVID-19. There was no significant change in the number of in-person interactions between the pre- and postimplementation phases for both nurses (5.7 vs 7.0 entrances per patient,
Telemedicine was rapidly adopted with the intent of minimizing pathogen exposure to health care workers during the COVID-19 pandemic, yet RTLS movement data did not reveal significant changes for in-person interactions between staff and patients under investigation for COVID-19 infection. Additional research is needed to better understand how telemedicine technology may be better incorporated into emergency departments to improve workflows for frontline health care clinicians.
The COVID-19 pandemic acutely raised concerns about the risks of nosocomial transmission of infection within the walls of health care systems. Several health care systems experienced outbreaks [
Telemedicine was one technology deployed by health care systems with the goal of protecting patients and staff during the COVID-19 pandemic, enabling clinicians to provide care remotely to both patients who are at home for outpatient evaluation and also on premises for acute care [
A real-time locating system (RTLS) uses a combination of sensors installed in defined locations and locators carried by staff or installed on equipment to track movement. These data may then be visible on a live monitor or recorded on a database log for further analysis, where it may provide operational or research value. For example, an RTLS can be used to measure and improve health care delivery [
We recognized an opportunity to evaluate whether an RTLS can quantify the impact of the rapid deployment of telemedicine on health care clinician workflows, measured by changes in movement through patient rooms. We aimed to understand (1) the feasibility of linking multimodal data, including RTLS data, to develop measures that summarize relevant physical workflows in a complex environment, and (2) describe the changes in in-person interactions between staff and patients observed after deploying a telemedicine platform in an emergency department early in the COVID-19 pandemic.
Stanford Health Care (SHC) launched an RTLS platform (Midmark) in conjunction with the opening of a new hospital building in late 2019. Infrared and radiofrequency sensors were installed in every patient care room, and staff were given RTLS badges to wear alongside their name badge [
At the start of the COVID-19 pandemic, SHC formed an operational committee to guide its telemedicine strategy. When the first patient was tested for COVID-19 on March 2, 2020, clinicians could activate a previously deployed communication technology (Cisco Jabber) that was primarily used for messaging between clinicians. While the technology did offer a videoconference feature, it required a staff member within the room to accept the call to initiate two-way communication. Given this limitation, a new platform composed of a consumer-grade tablet (Apple iPad) mounted on stands with wheels and a secure videoconferencing service (Zoom) was deployed in all patient care rooms throughout the emergency department [
Before entering the emergency department, patients were screened for any symptoms of influenza-like illness outside of the entrance. They were then triaged into one of three pathways: low-acuity patients meeting the screening criteria for COVID-19 were sent for testing in the garage outside of the emergency department (where the RTLS was not deployed), high-acuity patients meeting the screening criteria were sent to a designated waiting room isolated from the remaining emergency department, and patients not meeting the screening criteria were sent to the primary waiting room. Once inside the emergency department, every patient who was screened for COVID-19 was placed on appropriate isolation precautions and underwent laboratory testing for SARS-CoV-2. Clinicians were able to identify patients who were being screened for COVID-19 by isolation signage on the room entrance, an indicator on the electronic locator board, and from within the electronic health record. Clinicians were free to choose whether they would interact with patients with isolation precautions in person or via telemedicine. Nonetheless, normal standards of care were expected, including physical examination of patients during the encounter.
We identified multiple streams of data necessary to quantify the impact of telemedicine on health care worker movement. First, we extracted the RTLS event logs for movement of nursing and physician staff in each patient care room of the emergency department. RTLS data from staff in other clinical roles, such as radiology technicians and phlebotomists, were not included in this study as these roles were not the target users of the telemedicine technology. In addition, all staff members included in the study had dedicated clinical roles in the emergency department. We also obtained utilization logs from the telemedicine platforms. Finally, we queried our electronic health record system for the isolation status of all emergency department encounters that would indicate a patient needed screening for COVID-19, along with room information, time of rooming, and time of disposition decision. Patients who were never placed in isolation for COVID-19 testing during their evaluation were excluded from the analysis, as these patients were not the target of telemedicine usage.
The RTLS and encounter data were then processed and unified to provide measurements of the primary outcomes—the number of staff entrances per qualifying emergency department encounter and the total duration spent in the patient rooms over each encounter. First, when separate, successive RTLS events linking the same staff member and room number occurred within 30 seconds of one another, these events were combined into a single event. This assumption accounts for the possibility that the line of sight between the staff member’s badge and room sensor was briefly interrupted by a shift in the location of people or equipment inside the room. We conducted sensitivity analyses using thresholds of 1 second and 10 seconds and observed no significant difference in findings. The RTLS staff movement data was then merged with data on patient encounters using room names and timestamps extracted from the electronic health record system. Since patients with longer encounters in the emergency department are naturally more likely to have more interactions with health care clinicians, measures for staff entrances and duration per encounter were adjusted by encounter length of stay (time between initial rooming and disposition time). For example, to adjust the number of entrances for each encounter, the raw number of staff entrances was divided by that encounter’s length of stay and then multiplied by the average encounter length of stay observed over the entire study period. This method of adjustment allowed for data to be analyzed at different levels of granularity, including individual encounters, daily summaries, and over the study phase. In addition, as uptake of RTLS badges was different among physicians and nurses, we generally present statistics stratified by clinical role and use rates instead of absolute counts to make interpretation more consistent.
To describe telemedicine utilization, videoconference logs from both the Cisco and Zoom systems were summarized into daily activity reports for the entire emergency department, as the technology did not track which telemedicine sessions were linked to specific patient encounters. These daily summaries were divided by the daily emergency department census of patients screened for COVID-19 (derived from encounter data) to gather an approximate number of video conferences per patient encounter.
The final data set was then separated into the pre- and postimplementation phases to understand the change over time. To allow for a similar length of observation to the preimplementation phase from March 2 to March 28, data were collected for the postimplementation phase from March 29 to April 20, 2020. For each outcome, mean (SD) values for encounter-level measures were calculated for each phase, along with
Staff and patient data were deidentified prior to being available to the analytics team. This project was not deemed human subjects research by our Institutional Review Board (protocol #55927).
During the evaluation period, SHC cared for 6951 patients in its adult emergency department, of which 2662 (38.3%) were evaluated for COVID-19, with an average length of stay of 251 minutes. RTLS badges were worn by 245 unique staff members including 40 out of 99 (40.4%) attending physicians, 8 out of 62 (12.9%) resident physicians, and 197 (100.0%) nurses scheduled for service. Rates of badge use were similar between pre- and postimplementation phases (
Characteristics of patients, staff, and real-time locating system (RTLS) events in the emergency department stratified by study periods. Staff data report the number of staff wearing RTLS badges compared to those scheduled for at least one shift during that study phase.
Variable | Preimplementation phase | Postimplementation phase | |||
|
|||||
|
All patients evaluated, n | 4571 | 2380 | ||
|
Patients tested for COVID-19, n (%) | 1502 (32.9) | 1059 (44.4) | ||
|
Length of stay for patients tested for COVID-19 (minutes), mean (SD) | 190 (209) | 339 (299) | ||
|
|||||
|
Total staff | 220/328 (67.1) | 212/315 (67.3) | ||
|
Nurses | 182/182 (100.0) | 174/174 (100.0) | ||
|
Attending physicians | 32/96 (33.3) | 33/91 (36.3) | ||
|
Resident physicians | 6/50 (12.0) | 5/50 (10.0) | ||
|
|||||
|
RTLS events for COVID-19–tested patients | 5855 | 9886 |
Staff engaged in 892 videoconferences with patients screened for COVID-19 (0.6 videoconferences per patient) during the preimplementation phase using the existing Cisco platform and 1482 videoconferences (1.3 videoconferences per patient) using either the Cisco or Zoom platform during the postimplementation phase (
Nurses had an average of 7.3 (SD 6.9) (after adjustment for patient length of stay: mean 5.7, SD 4.1) entrances per patient encounter in the preimplementation phase. After implementation, nurses had an average of 10.8 (SD 10.2) (after adjustment: mean 7.0, SD 5.4) entrances per patient encounter, which was not a statistically significant difference between the phases (
In-person staff interactions with patients under investigation for COVID-19. Entrances into rooms with patients under investigation by nurses (daily census-weighted average over phase in dark blue, daily averages in light blue) had a nonstatistically significant increase between the pre- and postimplementation phases. Physicians (daily census-weighted average over phase in dark red, daily averages in light red) physically entered patient rooms much less often than nurses, and changes in physician entrances into patient rooms over phases were more subtle in absolute counts. A timeline of relevant public health events is provided below [
The combination of videoconferencing and RTLS data created a profile of the staff’s mode of interaction with patients. During the preimplementation phase, virtual interactions lasted an average of 4.3 minutes per encounter and accounted for 7.3% of total time staff spent with patients either in the room or via telemedicine (
Cumulative staff-patient interaction for patients under investigation for COVID-19. In-person exposure to patients under investigation was mostly borne by nurses (daily average shaded light blue) but did not change significantly over the phases. A similar trend was seen for in-person contact time for physicians (daily average shaded blue). Virtual contact (daily average shaded dark blue) with patients increased during the postimplementation phase, demonstrating adoption of the new telemedicine technology platform beyond the previously available platform. However, there was not a statistically significant difference in total interaction time with patients between the pre- and postimplementation phases (average for entire phase shown as a purple line).
The combination of multimodal data can capture meaningful workflow components of staff movement in an emergency department during the implementation of a new software technology, even in the course of a pandemic. RTLS data have been previously utilized to evaluate clinical workflows using process mining [
With the addition of a new telemedicine platform, we expected that virtual interactions would replace a portion of in-person contact with patients and thus theoretically reduce pathogen exposure and PPE use. For example, a reduction in the number of entrances into patient care rooms may result in a reduction in PPE use. However, after cross-validating telemedicine adoption data with RTLS data, we found no significant difference in either entrances or duration of in-person physical interaction across multiple clinical roles, including nurses, attending physicians, and resident physicians, suggesting exposure risk was not reduced despite an increased use of videoconferencing. We also found that the cumulative time clinicians spent interacting with patients either in person or virtually did not change significantly in the context of a pandemic and new workflows.
Not unexpectedly and consistent with prior findings [
In addition, other concurrent changes in technology and care standards may offer insights into why patient room entrances and time spent in person with the patient did not differ in the pre- and postimplementation of telemedicine technology. First, the development of a rapid SARS-CoV-2 RT-PCR (reverse transcription–polymerase chain reaction) assay early in the postimplementation phase may have allowed physicians to delay in-person evaluations of patients until they received test results, permitting clinicians to modify their routines depending on perceived risk after a negative COVID-19 result. Second, as emergency department staff became more comfortable with isolation protocols, they may have felt less fear of contracting the disease, making them more comfortable spending time in person with patients even when a given clinical activity, such as decision-making or counseling, could have been done via telemedicine. Lastly, during the early period of the pandemic when PPE resources were scarce, nurses may have also consolidated tasks to minimize entrances into isolation rooms. These behavioral and workflow changes may have reverted to baseline levels as the PPE inventory stabilized. However, there was a gap in the timeline between these events both before and after the telemedicine deployment, and telemedicine adoption was greatest immediately after deployment. Thus, the likelihood of observing a significant change due to the technology was high if that signal truly existed. The lack of significant differences in room entrances and total time spent with patients between the pre- and postimplementation periods may simply be a reflection of this particular technology deployment at a specific phase of the pandemic response, characterized by simultaneous and often transient complexities. Most importantly, an RTLS-based analysis can serve as a useful template for future evaluations of health technologies during emergencies, particularly if a contemporaneous comparator can be found.
This evaluation has certain limitations. The RTLS data and telemedicine data were unable to be merged by patient room, limiting our understanding of how staff used telemedicine and how workflow was affected for specific patient groups. Understanding the differing use of telemedicine by nurses and physicians (which was not available in our study) may be a fruitful area for future research, particularly if such data can be matched to individual staff movement data. Finally, as both attending and resident physicians did not universally wear RTLS badges, the physicians included in our analysis may not be representative of others who chose not to wear an RTLS badge. A clear strength of the evaluation, however, is that movement data was available for all nursing staff within a single emergency department and was able to be linked to individual patient encounters.
Movement data captured by an RTLS is a rich technological resource that can assist in monitoring the impact of changes in workflow even in a rapidly changing clinical environment. We were able to leverage an RTLS to quantify staff movement in our emergency department as we deployed a new telemedicine platform during the COVID-19 pandemic. While telemedicine did see a significant increase in adoption in the emergency setting, it did not ultimately influence physician and nursing movement in and out of the rooms of patients who were under investigation for COVID-19. These findings underscore the need for additional formal evaluations to determine whether informatics interventions (and the significant resources directed to their deployment) are having their intended impact on health care worker safety. Such operationally relevant analyses can be enabled by unifying real-world data from multimodal platforms.
personal protective equipment
reverse transcription–polymerase chain reaction
real-time locating system
Stanford Health Care
The authors would like to thank Rajiv Ramdeo, Ann-Marie Yap, Kevin Yuen, Derek Nielsen, Sam Shen, Gretchen Brown, and Steven Asch for their contributions to this project.
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. BP and SV were funded through the Clinical Informatics Fellowship and the Stanford-Intermountain Population Health Fellowship, respectively.
All authors made substantial contributions to conception and design as well as acquisition of data or analysis and interpretation of data; were involved in drafting the manuscript or revising it critically for important intellectual content; gave final approval of the version published; and agreed to be accountable for all aspects of the work.
None declared.