Sepsis is the leading cause of death in US hospitals [1], accounting for 6% of all hospitalizations and 35% of all inpatient deaths [2]. Hospital performance in sepsis care has a significant impact on patient outcomes. A recent study demonstrated that every hour of delay in the completion of a sepsis bundle, including antibiotic administration, was associated with a 4% increase in risk-adjusted hospital mortality [3]. Thus, every failure to complete a sepsis bundle on time potentially represents a barrier to optimal patient care. There is a clear and urgent need for effective interventions to improve the performance of sepsis care and meet the expectations of patients and stakeholders.

International guidelines and the Centers for Medicare and Medicaid Services (CMS) Sepsis Core Measures (SEP-1) bundle emphasize the importance of adhering to specific steps in the diagnosis and management of sepsis [4]. Consistent adherence can be very challenging, especially in the setting of a busy emergency department (ED), ward, or intensive care unit, where there are multiple simultaneous demands on providers’ attention. In this environment, SEP-1 bundle care can be delayed or missed, even when team members are knowledgeable about best practices in sepsis care [5].

The CMS SEP-1 severe sepsis bundle is an all-or-nothing measure that requires antibiotics, blood cultures, and a lactate measurement within 3 hours of sepsis onset and a repeat lactate measurement within 6 hours if the initial lactate level is elevated. In cases of septic shock, additional requirements include adequate fluid resuscitation, reassessment of volume status and perfusion, and possible addition of vasopressors. As an all-or-nothing measure, a deficiency in any element of the bundle counts as failing the entire measure. In the first 2 years after the measure was introduced in 2015, two-thirds of sepsis cases submitted to CMS by hospitals failed the measure [6], and recent Hospital Compare data from 2019 still show a failure rate of 61% [7]. There has been argument in the literature about the appropriateness of some components of the bundle [8-10]; however, the fact remains that early identification and treatment of patients with sepsis remains an important quality opportunity.

Institutions have attempted both nondigital and digital processes to improve sepsis care, including provider and nursing huddles and standardized electronic order sets. Most digital initiatives studied thus far have focused on identifying patients at risk for sepsis using the severity of illness scores in the electronic medical record (EMR). This has helped improve the early detection of sepsis [11,12] but has only modest effects on adherence to evidence-based bundle care elements and outcomes [13]. In concept, digital solutions offer the potential to close this gap by enabling real-time monitoring of adherence and providing just-in-time alerts to providers at the bedside. Electronically available data from the patient’s medical record would make this possible, as it includes most of the key elements of the SEP-1 bundle, including venipuncture for laboratory studies and medication administration. Any solution that could increase adherence and decrease cognitive load must also ensure that it does not cause alarm fatigue.

To date, we have not found any studies in the literature describing electronic alerting based on the completion of bundle elements for sepsis. Therefore, we developed a digital software solution called the Sepsis Care Tracking Platform (SCTP), which continuously monitors patients at risk of sepsis for completion of SEP-1 bundle elements and delivers actionable reminders to medical providers at the bedside in time for them to correct these deficiencies.

The objective of this paper is to describe the design, implementation, and validation of a novel electronic monitoring platform for optimal sepsis care in an ED at a large urban teaching hospital. By establishing feasibility, we hope to lay the groundwork for future randomized controlled studies that can determine the efficacy of the electronic monitoring platform in impacting process and outcome measures in sepsis care.

Multiple stakeholder groups were consulted to devise and implement the SCTP in the ED. For project approval and scope, the study team met with the chairperson and operational leaders of the Department of Emergency Medicine. The project was also presented to the quality and safety committees, both at the departmental and hospital levels. For feedback and iterative improvement, the ED Quality and Safety Committee was updated every 2 weeks regarding the project status and areas for improvement. ED providers who were paged by the SCTP received surveys regarding their experience and suggestions for improvement. This study was a quality improvement project based on institutional guidelines from the institutional review board and was thus exempted from the institutional review board review.

The timeline for this project was divided into three phases: design and technical development, pilot, and postpilot program rollout.

The SCTP ED platform was coded in C#, using .NET Microsoft Development Platform and Microsoft SQL databases. It was linked to the institutional EMR (EPIC) through a web service that extracted data each minute. The SCTP checked for the completion of the sepsis bundle elements every 15 minutes. These elements included completion of blood culture orders; initial and repeat lactate results; and administration of antibiotics, vasopressors, and fluids (as described in the Results section, vasopressors and fluids were later eliminated from the SCTP). If one or more elements were incomplete, the ED providers would be paged with the deficiency. Each patient was added to the platform after the institutional sepsis best practice alert (BPA) was accepted and monitored until the end of the 6-hour window for optimal sepsis care. The platform tracked a patient within that window even if the patient was admitted or transferred to a different area of the hospital and continued to page the appropriate responding clinician.

During both the pilot and postpilot program rollout phases, several study measures were tracked to assess the platform function and outcomes. Data were abstracted from the SCTP, an institutional clinical and administrative database, and surveys administered to paged providers. Subsequent data analyses were performed using Microsoft Excel.

We defined an alert as when the SCTP identified a sepsis bundle element deficiency within 1 hour of the bundle element becoming due. We defined a page as occurring when a clinical provider received a notification on their pager. As there were usually multiple providers caring for each patient, a single alert could trigger multiple pages at the same time. During the pilot phase, we assessed the total number of pages sent to providers; the number of alerts per provider shift, defined as the number of alerts divided by the number of shifts completed by the attending providers; and the etiology of alerts, defined as the bundle measure that triggered the alert. In addition, we administered an email survey to the providers after receiving an alert. The survey consisted of two questions regarding whether the pages were helpful and whether they had any further feedback regarding the SCTP.

During the postpilot program rollout, we measured the characteristics of patients who were monitored by the platform, including age and length of stay in the ED. Similar to the pilot phase, we also measured the number and etiology of the pages sent to the providers. In addition, we measured the postalert successful adherence rate of sepsis bundle measures, which was defined as the completion of the sepsis bundle element within the next hour after the relevant page was sent to the provider.

The SCTP was developed and validated in close cooperation with ED Quality and Safety leadership and intended to fit within the existing workflow of the institution to decrease barriers to adoption.

In consultation with ED Quality and Safety leadership, it was determined that patients should qualify for monitoring if they met the criteria for possible sepsis based on an electronic health record (EHR)–derived algorithm that existed before this study. This algorithm is based on the Sepsis-3 2016 European Society of Intensive Care Medicine and Society of Critical Care Medicine consensus definition of sepsis and incorporates both signs of infection such as fever, positive laboratory results, and patient risk factors, as well as signs of organ dysfunction as measured by the Sequential Organ Failure Assessment score [14]. Providers caring for patients identified as potentially septic receive a prompt in the EHR in which they could click that they agreed or disagreed that sepsis was likely. If accepted, the patient would qualify for SCTP monitoring. Stakeholders and study team members believed that the advantage of this approach was to include only those patients in the active monitoring cohort whom the team suspected of sepsis, thereby increasing the relevance of SCTP alerts they received (Figure 1).

From the commonly accepted SEP-1 bundle, stakeholders selected antibiotics, blood cultures, and lactate measurements as the bundle elements that would be monitored and excluded measures related to the volume of intravenous fluids administered. The decision to exclude fluids was made because of the belief that significant practice variation existed due to patient-specific conditions and comorbidities [15,16] and that alerts suggesting that additional fluids should be administered could be clinically inappropriate.

Stakeholders agreed that paging 1 hour before the deadline for bundle compliance gave providers sufficient time to address the deficiency (Figure 1). To minimize alert fatigue, stakeholders agreed that providers would receive a maximum of two pages per patient that would be sent 1 hour before the 3-hour and 6-hour SEP-1 time windows. Each page would list all the deficient bundle elements at that time.

Given that multiple providers were simultaneously responsible for a single patient, stakeholders designed the software to use the logic shown in Figure 2 to determine the appropriate clinicians to page. The decision was made to send alerts to both the responding clinician (resident or advanced practitioner) and the supervising attending because each played a unique role in the patient’s sepsis care. The resident and advanced practitioner were responsible for placing orders and closing the loop of communication with bedside nurses about bundle measures, whereas the attending physician could supervise and monitor the overall care of the patient to ensure appropriate care was provided. Although the study team and ED stakeholders thought it valuable to alert bedside nurses, given their important role in sepsis care delivery, this was technically not feasible as they did not carry pager devices.

Stakeholders agreed that alert message contents should state the deficient element without providing prescriptive language recommending that the provider performs a particular action (Figure 3). The message contents would list which element was deficient, but otherwise the remainder of the message would remain the same for all pages. This decision was made on the rationale that in some cases providers would appropriately not administer bundle care to patients based on the evolving diagnostic and clinical course of the patient.

Stakeholders determined that the hospital’s paging system was the most effective means of communicating time-sensitive alerts, given that providers always carry their pager and typically view messages immediately. This was preferred over EMR-based pop-up alerts because it required the provider to log on to the patient’s medical record to see the alerts. It was felt that this mode would be burdensome to the provider and potentially delay the receipt of the message.

During the 35-day period, 453 patients met the inclusion criteria to be monitored by the SCTP. Overall, 371 pages were sent, resulting in 0.79 pages per attending shift. Of the alerts, 27.3% (50/183) were for blood cultures, 50.3% (92/183) were for antibiotics, and 22.4% (41/183) were for lactate measurements.

In total, 70 surveys were returned with responses. In total, 87% (61/70) of respondents found the alerts to be helpful (21/70, 30%), very helpful (17/70, 24%), or extremely helpful (23/70, 33%). Specific examples of positive feedback included “The page helped us realize blood cultures were not initially ordered and we ordered them after this page” and “It prompted us to reassess the patient for sepsis.” Providers also suggested that pages “would be more useful if patient’s name was also displayed on the page and not only the MRN.” Providers also noted that their suspicion for sepsis can change over the course of an ED visit, with comments such as “Patient was later felt not to have sepsis so bundle was not completed.” On the basis of this feedback, the alert message content was modified to allow providers to more easily identify the patient (Figure 3) and the alert routing algorithm was adjusted to identify the correct attending physician (Figure 2). We also removed reminders to initiate vasopressors for hypotensive patients because a reliable bedside monitoring alarm system already existed to alert clinicians to hypotension and clinicians consistently responded by initiating vasopressors when appropriate. Our survey responses helped to elucidate this frontline practice.

After completion of the pilot program with subsequent adjustments to the SCTP, the postpilot program rollout phase was initiated. During this 48-day period, 711 patient encounters (mean 14.8, SD 4.4 encounters per day) for 648 unique patients met the inclusion criteria to be monitored by the SCTP (Table 1).

Of the 711 encounters monitored by the SCTP, 439 (61.7%) passed all measures without requiring a page, as all sepsis bundle elements were completed more than an hour before the deadline. In comparison, 38.3% (272/711) patients had one or more pages sent to the responding providers. In total, 634 pages were sent, averaging 0.89 pages per patient in the active monitoring cohort and 0.98 pages per attending shift (Table 2).

Of the 634 pages, the most frequent alert sent was for deficient antibiotics (136/327, 41.6%), followed by the second lactate measurement (106/327, 32.4%), blood cultures (68/327, 20.8%), and first lactate (17/327, 5.2%). This distribution was similar to the results from the pilot study, with pages related to antibiotics being the most common etiology.

Overall, there was a substantial proportion of sepsis bundle element deficiencies that triggered a page and were successfully adhered to within the SEP-1 time window. There was a 26% (18/68) postalert successful adherence rate for collecting blood cultures, 12.5% (17/136) for administering antibiotics, and 24% (4/17) for initial lactate measurement and an impressive 81.1% (86/106) for the second lactate measurement.

To improve adherence to sepsis guideline care at our tertiary care hospital, we designed and implemented a real-time sepsis care tracking and alerting platform in a busy ED environment. With feedback from frontline clinicians, quality and safety program representatives, and departmental leadership, we customized the SCTP to the needs of clinicians and the clinical environment. In our postpilot program rollout phase, the SCTP actively monitored 711 ED encounters where sepsis was thought to be likely, in 272 of these encounters, the platform sent text pages to ED providers due to a deficiency in sepsis bundle completion, with an overall postpage adherence rate of 38.2% (125/327).

During the development phase, it was important to select the sepsis bundle elements that were felt to be most clinically appropriate and could be identified by our monitoring platform. The SEP-1 framework served as a foundation for this study, as the framework was widely accepted and aligned the interests of multiple stakeholder groups, including providers, departmental and quality and safety leadership, and hospital administration. Engaging with provider stakeholders allowed us to make the application more effective. Through these discussions, we were able to understand the clinical workflow and target alerting to the frontline provider who places the orders.

Moreover, we note the importance of the provider feedback obtained during the pilot phase. The surveys emphasized the need to provide critical information concisely in the SCTP paging alerts, including what information can quickly and accurately identify the patient and the specific bundle element deficiency. In addition, given the complex structure of provider teams in our ED, which can change multiple times over a patient’s ED visit, we had to consider different clinical scenarios and which provider would be the optimal recipient for paging alerts. Finally, given the learning curve that comes with new technologies, the pilot phase allowed providers to acclimate to this system before the full rollout.

During the postpilot rollout, we found that there was a substantial proportion of encounters (272/711, 38.3%) in which the clinician was at risk of missing the SEP-1 care measure and for which an alert was paged, indicating the potential impact of such a platform on adherence. Furthermore, we found that the postalert successful adherence rate ranged from 12.5% (17/136) to 81.1% (86/106), depending on the measure for which the alert was sent, further suggesting the potential for the platform to improve adherence. Finally, we found that the number of pages per clinician shift (0.98) was low enough to not be overly disruptive but high enough that we would expect the SCTP to meaningfully impact SEP-1 performance. On the basis of these findings, we believe that a real-time sepsis care tracking and alerting platform is feasible in a hectic ED environment and could have a significant impact on adherence to life-saving sepsis care without causing alarm fatigue.

In addition, we found it concerning that the most common deficiency triggering a paging alert was a delay in antibiotics, which is the SEP-1 element that is most closely linked to mortality in previous studies [17]. This suggests the possibility that the SCTP could be associated with mortality improvement if it is able to improve timely antibiotic administration. However, we found that the postalert adherence success rate for antibiotics was the lowest among all alerts. It may be that antibiotics were, in fact, administered just outside of the monitored 3-hour window or that patients were subsequently determined to likely not have sepsis. Regardless, this discrepancy requires further exploration to determine why antibiotics may have a lower rate of postalert adherence compared with other measures.

In comparison, we found that repeat lactate was the second most common etiology of pages (106/327, 32.4%) and had the highest postadherence rate (86/106, 81.1%). This suggests that the SCTP may be especially valuable over a prolonged time course in which providers may be distracted by the needs of other patients, and paging alerts in this situation can refocus provider attention with sufficient time to deliver optimal care.

We believe that the postalert adherence rate of 12.5% (17/136) to 81.1% (86/106) for bundle element deficiencies represents the completion of bundle elements that likely would have been missed otherwise and is a significant potential improvement in increasing adherence to sepsis bundle care. However, the majority of pages did not result in adherence, which deserves further study. On the basis of our survey data, we believe that an important driver of this trend may be that the clinician no longer believed sepsis was likely. The sepsis BPA, which the SCTP used as a trigger to begin monitoring, was designed to cast a wide net early in the patient’s ED encounter to reduce the likelihood of missing any patient who could be septic given the high mortality of the disease. As more data are gathered and the patient’s care evolves, clinical teams may have judged that sepsis was unlikely and thus appropriately withheld sepsis bundle care. We believe that this could have accounted for much of the nonadherence. A future iteration of the SCTP may enable frontline clinicians to off ramp from the page alerts if they judge that sepsis is highly unlikely later in the patient’s ED course.

SCTP differs from previous EHR-related innovations in improving sepsis care. The majority of previous studies have focused on identifying sepsis through different clinical criteria [18] or through predictive algorithms and machine learning [19], usually followed by an EHR warning or paging alert. Other studies have used alerts to trigger sepsis workflows, including bedside assessment and order sets [11]. SCTP builds on these innovations by following patients suspected of having sepsis through the next several hours of care, which can be the most critical and prompts providers toward optimal care. Therefore, we expect it to influence provider action more directly for patients suspected of having sepsis and have a meaningful impact on clinical outcomes.

The focus of this study is on alert fatigue, as recent articles have shown that an increase in the number of alerts decreases the provider’s acceptance rate [20] and many alerts in the EHR may not directly change clinical care [21]. The SCTP incorporated many design features to minimize alert fatigue, such as restricting enrollment to patients where the sepsis BPA was confirmed by an ED provider and condensing pages so no more than two pages are sent for any patient. Consultations with frontline providers and survey data were also used to optimize the message content to determine the appropriate page recipient. All of these factors likely contributed to the low number of pages sent per attending shift (0.98) and favorable survey responses. However, further work is needed to ensure that the clinical benefit of this system outweighs the alert fatigue it generates.

This study had several limitations. First, system implementation was conducted at a single academic site, which may not be generalizable across different health care settings. Second, the survey response rate was low for providers receiving alerts from the paging system. Subsequent iterations may consider text-based phone messaging or in-person surveys at the time of the provider receiving the page, which may encourage more detailed and complete responses. Third, the study was not designed to measure the effect size of SCTP paging alerts or review patient cases where providers understand sepsis to be unlikely despite the initial BPA. Our institution is currently initiating a randomized controlled trial to determine the efficacy of SCTP in improving compliance with SEP-1 bundle elements and patient outcomes. It will also further quantify any marginal increase in alert fatigue. Finally, there may be other simultaneous quality improvement initiatives and the Hawthorne effect that may confound our data.

In conclusion, we have demonstrated a feasible, technically sound, real-time provider alert system that has the potential to improve the SEP-1 compliance rate without significant amounts of alert fatigue. We anticipate that future work will involve a randomized controlled trial to measure potential increases in SEP-1 compliance rates and patient outcomes and further characterize alert fatigue. The expansion of this promising intervention could be considered in other inpatient settings.