Sepsis, defined in 2016 as “life-threatening organ dysfunction caused by a dysregulated host response to infection,” is a leading cause of death worldwide [1]. A recent study by Rudd et al [2] estimated that 48.9 million cases of sepsis were reported in 2017, with 11 million sepsis-related deaths, representing 1 in 5 of all deaths globally [2]. Furthermore, survivors of sepsis often have a decreased quality of life, including higher rates of mortality, physical disabilities, chronic illnesses, mental health issues, and cognitive impairments [3-9].

Prompt administration of sepsis therapies, such as intravenous antimicrobials and fluid resuscitation, is associated with better patient outcomes and lower health care–related costs [10,11]. Therefore, it is critical to detect sepsis as early as possible to ensure rapid initiation of treatment [12-14]. Unfortunately, sepsis has no diagnostic gold standard and extremely heterogenous signs and symptoms, making it difficult for clinicians to distinguish it from other acute conditions [15]. The use of sepsis identification tools, such as the Quick Sepsis-Related Organ Failure Assessment, the National Early Warning Score, and the Adult Sepsis Pathway, helps facilitate early sepsis recognition [16-18]. However, these tools typically rely on manual input of vital sign information and score calculation by clinicians. Thus, timely sepsis identification hinges on vigilant and regular patient monitoring [19]. These difficulties often result in delayed sepsis diagnosis and treatment in hospitals [19,20].

The extensive implementation of data-rich electronic health records in health institutions has brought the opportunity for widespread integration of digital health care support systems [21]. In particular, the incorporation of computerized clinical decision support (CCDS) into hospital systems has the potential to assist accurate and timely early sepsis detection. CCDS systems can be designed with integrated sepsis-risk warning tools that alert clinicians to patients at risk of sepsis [13,22], reducing the physical and mental workload associated with manual patient monitoring [21].

Over the past 10 years, CCDS technology has rapidly expanded, with two distinct approaches emerging: knowledge-based CCDS systems programmed with predefined rules derived from established clinical knowledge and adaptive CCDS systems using artificial intelligence and machine learning techniques [21,23,24]. In this scoping review, we focused on the use of knowledge-based CCDS systems in sepsis detection.

The use and implementation of sepsis CCDS systems in real-world clinical settings is a novel, rapidly expanding, and highly complex field [21,25]. In this scoping review, we systematically mapped the literature available on sepsis CCDS systems with the intention of identifying knowledge gaps and informing future research. The research question directing this review is What is the evidence base for the use of knowledge-based clinical decision support systems in hospitals for early sepsis detection and how have they been evaluated?

More specifically, through this scoping review, we aim to (1) scope the study contexts, designs, and research methods used; (2) summarize the study outcomes investigated; and (3) map the range of CCDS system designs and implementation features, such as sepsis clinical criteria.

The detailed methodology for conducting this scoping review was published previously in a protocol [26]. In brief, the review was guided by the Joanna Briggs Institute Reviewer’s Manual [27], the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) [28], and the 5-stage scoping review framework proposed by Arksey and O’Malley [29]. A search for current reviews and protocols on this topic was undertaken and confirmed the absence of scoping reviews. A completed PRISMA-ScR checklist is attached in Multimedia Appendix 1 [28].

We used a broad 3-step search strategy, as outlined in our protocol [26]. An experienced librarian was consulted to help construct and refine the search. The final search strategy combined terms relating to sepsis with CCDS and detection, while excluding artificial intelligence, and was used to search MEDLINE, Embase, CINAHL, the Cochrane database, LILACS (Latin American and Caribbean Health Sciences Literature), Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT (ProQuest Dissertations and Theses Global). We restricted the search to human studies in the English language. An example of the final strategy adapted for MEDLINE can be seen in Multimedia Appendix 2. The database search was undertaken in September 2020, with no date limits applied. The reference lists of relevant systematic reviews were hand-searched to identify additional studies. Any studies identified via hand searching up until the end of data extraction (early 2021) were included. We included both peer-reviewed journal articles and gray literature (ie, conference abstracts and theses).

Following the search, duplicates were removed as was gray literature that had been published as a peer-reviewed journal article. However, we kept studies if they reported the same methods and study cohort but examined different outcomes. Using the eligibility criteria as reported in our protocol [26], 2 reviewers (KA and JB) independently performed title, abstract, and full-text screening, with any disagreements resolved through discussion or review by a third researcher (LL). Title and abstract screening was piloted with a random selection of 25 studies by both reviewers (KA and JB). Similarly, full-text screening was piloted with a random selection of 10 studies. The 2 reviewers (KA and JB) had 100% agreement during the title and abstract screen pilot, 97.6% agreement for the full title and abstract screen, 60% agreement for the full-text screen pilot, and 77.4% agreement for the full-text screen. Hand searching was completed by 1 reviewer (KA) with identified studies confirmed by a second (JB). A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram visually illustrating this process is shown in Figure 1.

Following screening, it was determined that the results of this review would be split over 2 papers, one investigating adult or unspecified populations and another investigating pediatric, neonatal, and maternal populations. Pediatric, neonatal, and maternal populations have remarkably different sepsis presentations and physiology compared with the general adult population [30-32]. The separation of results will allow for a more meaningful analysis. Included studies with unspecified age were assumed to likely include all patients in a general hospital setting and were grouped with adult populations. This paper reports the results of all studies investigating CCDS systems studied in adults or populations with an unspecified age.

The data charting form used was iteratively designed based on the study aims. The form was piloted by a single reviewer (KA) and double-checked by a second (JB). Changes to the form were made following discussion between 3 reviewers (KA, JB, and LL). Data charting was performed by 1 reviewer (KA) with ongoing consultation with the review team.

The final data charting form included the components listed in our protocol [26], with minor adjustments as reported in Multimedia Appendix 3 [33-39]. Notably, an additional category clarity of outcome reporting was added to the form to account for the variability in outcome reporting transparency. Studies were categorized as having good clarity of outcome reporting if they specified the primary outcomes, the outcome analysis method, and the outcome measure definitions and poor clarity if the outcomes were not clearly described or there was a substantial reporting discrepancy between the methods and the results. Studies were categorized as having average clarity if they fulfilled some criteria of both good and poor.

We accepted any definition of charted data items as specified by the studies. For example, we accepted any definition of systemic inflammatory response syndrome (SIRS), any definition of sepsis, or any cost outcomes specified for the CCDS system. We defined the usability outcome category to follow the ISO definition of usability from ISO 9241-11:2018, section 3.1.1: “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specific context of use” [33]. We required usability outcomes to be specifically investigated from the perspective of end point users (ie, clinicians). To match this definition change, usability outcomes were retrospectively categorized into the effectiveness, efficiency, or satisfaction of the CCDS system from the user’s perspective.

The results were analyzed through both a narrative review and quantitative descriptive analysis. A narrative summary of the data is presented, organized by our 3 aims. The data charted for each aim are summarized into tables using frequency counts and percentages. Graphical figures were also produced, where appropriate.

Owing to the extensive scope of the data charted, several subgroups were collapsed into larger groups to avoid issues of small cell size and to allow for a more meaningful summary. A complete list of the smaller subgroups condensed into the larger groups, organized by table and figure, can be found in Multimedia Appendix 4. Of note, nurses were frequently reported as CCDS system responding personnel and so were grouped separately from other clinicians to better highlight this.

Ethical approval or consent to participate was not required for the scoping review. The data were charted from published studies, and no individual information was included.

Our initial search identified 22,190 studies, with 12,139 remaining after duplicate removal. Following title, abstract, and full-text screening, 149 studies met our inclusion criteria (Figure 1). Hand searching identified 10 additional studies, resulting in a total of 159 included studies. Of these 159 studies, 124 investigated adult or unspecified populations and were included in this manuscript (Figure 1). A table detailing the main study characteristics for all 124 included studies can be found in Multimedia Appendix 5 [40-163]. In total, 52.4% (65/124) of the studies were categorized as journal articles, 43.5% (54/124) as conference abstracts, and 4% (5/124) as theses (Multimedia Appendices 4 and 5).

The context and design characteristics of the studies included in this review are presented in Table 1. Of the 124 included studies, 111 (89.5%) used purely quantitative methods to evaluate CCDS systems (Table 1). Most studies (96/124, 77.4%) used either single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) study designs (Table 1). Very few studies used more robust study designs, such as randomized controlled trials (5/124, 4%), controlled studies (7/124, 5.6%), or interrupted time series (4/124, 3.2%; Table 1). None of the studies reported the use of reporting guidelines. An approximately even distribution of studies was observed across different hospital settings, such as hospital-wide, and specific settings (eg, intensive care unit [ICU], emergency department, and inpatient wards; Table 1).

All studies but 1 (123/124, 99.2%) were published from 2009 onwards, and of the journal articles, 85% (55/65) were published in 2014 or later (Figure 2). Overall, the number of journal articles published steadily increased over time. Of the 65 journal articles, 46 (71%) reported studies conducted in the United States; 2 (3%) each in Germany, Saudi Arabia, and the United Kingdom; 1 (2%) each in Australia, Brazil, Israel, and South Korea; and 9 (14%) did not report which country they were conducted in (Multimedia Appendix 6).

Just over half (66/124, 53.2%) of the studies specified the age of the population as adult. Within these studies, there was a reasonable variation in the actual age range provided. Almost half (29/66, 44%) reported an adult population aged ≥18 years, whereas 30% (20/66) of the studies did not specify an age range further than adult. The remaining studies reported populations using thresholds such as aged >14 (1/66, 2%), ≥14 (3/66, 5%), >16 (2/66, 3%), ≥16 (3/66, 5%), ≥19 (6/66, 9%), and ≥70 (1/66, 2%) years, with 2% (1/66) of the studies inconsistently listing multiple thresholds.

The outcomes investigated by the included journal articles and the conference abstracts and theses are presented in Table 2 and Multimedia Appendix 7, respectively. Of the 4 predefined outcome categories, patient outcomes were reported in the highest number of studies (107/124, 86.3%; Figure 3). Sepsis treatment and management outcomes were reported in 60.5% (75/124) of the studies, CCDS system usability outcomes in 11.3% (14/124), and cost outcomes in 7.3% (9/124; Figure 3).

Overall, only 31.5% (39/124) of the studies had good clarity in outcome reporting (Figure 4). Generally, studies had average (62/124, 50%) or poor clarity (23/124, 18.5%). Unsurprisingly, journal articles had better clarity of outcome reporting, with 40% (26/65) of the articles having good clarity, compared with 22% (13/59) of the conference abstracts or theses (Figure 4).

In the 65 journal articles, mortality was the most frequently reported patient outcome (39/65, 60%). Overall, 35 different types of mortality measures were reported 55 times across 39 studies (Multimedia Appendix 8). Of these, in-hospital mortality was the most frequently reported (13/55, 24%; Multimedia Appendix 8). Sepsis identification, length of stay, and other patient outcomes were also frequently reported, appearing in 38% (25/65), 34% (22/65), and 35% (23/65) of the articles, respectively (Table 2; see Multimedia Appendix 4 for the expanded list of included outcomes). ICU admission was the least reported patient outcome (12/65, 18%). In the sepsis treatment and management outcome category, antibiotic-related and other were the most frequently reported outcomes in journal articles (27/65, 42% and 31/65, 48%, respectively), followed by lactate-, fluids-, and blood culture–related outcomes (17/65, 26%; 14/65, 22%; and 14/65, 22%, respectively; Table 2; see Multimedia Appendix 4 for expanded list of included outcomes). Overall, sepsis bundle or protocol compliance was the least reported outcome in journal articles (12/65, 18%).

CCDS system usability outcomes were reported in similar numbers of journal articles, with 12% (8/65) of the journal articles reporting on the efficiency of the system, 11% (7/65) on system effectiveness, and 11% (7/65) reporting on users’ satisfaction with the system (Table 2). Among the CCDS system-related cost outcomes, cost was reported in the greatest number of journal articles (5/65, 8%), whereas cost-effectiveness or savings were reported in only 5% (3/65) of the articles (Table 2).

The characteristics of the CCDS systems reported in the included studies are presented in Table 3. Half (64/124, 51.6%) of the studies, most of which were journal articles (44/64, 69%), implemented homegrown CCDS systems. Of the 124 studies, only 13 (10.5%), including 10 (77%) journal articles, implemented commercial CCDS systems, of which 69% (9/13) were the St John’s Sepsis Surveillance Agent (Cerner Corporation, Kansas City, Missouri, United States; Table 3). Most included studies (95/124, 76.6%) evaluated live CCDS systems only, where the CCDS was implemented and actively sending alerts. Silent CCDS, where the system would run in real time but not send clinical alerts, were implemented by 7.3% (9/124) of studies, and 11.3% (14/124) of studies implemented both silent and live CCDS, either sequentially or concurrently.

SIRS alone was the most frequently used CCDS clinical criteria for sepsis identification (50/124, 40.3%), followed by SIRS combined with organ dysfunction (28/124, 22.6%), and SIRS combined with other criteria (23/124, 18.5%; Table 3). In addition, a diverse range of other criteria were used by 32.3% (40/124) of studies (Multimedia Appendix 4), while 7.3% (9/124) did not specify the clinical criteria used (Table 3)

Over half of the studies reported the implementation of CCDS systems alongside numerous other related interventions (68/124, 54.8%), such as staff education programs and antibiotic order sets (Table 3). The most common type of concurrent intervention used was clinical protocols in 41.9% (52/124) of the studies (Table 3).

Most commonly, studies reported nurses (51/124, 41.1%) or other clinicians (37/124, 29.8%) as the main CCDS alert responding personnel (Table 3). Some studies reported on CCDS with response teams (12/124, 9.7%), study coordinators (8/124, 6.5%), or other personnel (11/124, 8.9%) responding to the alerts. Of the 124 studies, 33 (26.6%) reported the use of the electronic health record to distribute CCDS alerts, 26 (21%) the use of pagers, 15 (12.1%) the use of a patient dashboard or work list, and 8 (6.5%) the use of another form of alert delivery (Table 3).

This review canvassed 124 studies in total, representing a comprehensive overview of current research, including an extensive body of gray literature. Over half of the included studies were journal articles (65/124, 52.4%), and nearly all studies were published in the last decade, indicating the considerable volume of recent research investigating the use of CCDS systems for early detection of adult inpatients with sepsis. Our findings demonstrate the substantial diversity of studies across all three aims: (1) the context and design of the study, (2) the type and measurement of outcomes investigated, and (3) the design and implementation of the CCDS system evaluated. We identified little research into the effects of CCDS on patient morbidity or CCDS usability and cost outcomes, highlighting key knowledge gaps in the literature. Our review also underlines the need for robust study designs, as well as improved generalizability and reporting in future studies.

There is extensive heterogeneity in the current literature investigating the implementation and evaluation of CCDS systems for early sepsis detection in adult hospital patients. In particular, there was considerable diversity displayed in the chosen clinical criteria for sepsis identification across the studies included in our review. Although many studies used the SIRS criteria, alone or with adjuncts, there was a substantial range of other criteria used (Multimedia Appendix 4). This can be attributed to the extremely diverse presentations of patients with sepsis, which has led to the development of numerous different clinical scores for sepsis detection [15-18,164]. In addition, our findings demonstrated variability in the method of alert delivery, personnel who respond to alerts, and concurrent implementation of related interventions. Studies were conducted across a range of different hospital settings, including hospital-wide or specific sites, such as the emergency department or ICU (Table 1). The chosen threshold for what age participants were included in the study was also quite variable, with studies defining their adult population using cutoff points ranging from 14 to 19 years and older. Finally, our review illuminated the expansive number of outcomes used to evaluate and investigate sepsis CCDS systems. Previous systematic reviews have similarly highlighted this diversity [13,21,22,165,166]. This heterogeneity across settings, participants, CCDS system characteristics, and outcomes makes it difficult to compare studies and to make general statements regarding sepsis CCDS systems.

This diversity can be partially attributed to the novel nature of sepsis CCDS systems and the recent emergence of the field. Our findings show a vast expansion of the literature, with three-quarters of studies published since 2014 (Figure 2). Owing to this recent rapid development of the field and the simultaneous evolution of information and communication technology in health care [21,167], there is no well-established research strategy or dogma for this specific area. Consequently, different authors have designed and executed their studies using a diverse range of variables and study design methodologies.

This variability can also be attributed to the complexity involved in the implementation of health care interventions [168,169]. To characterize this complexity, Greenhalgh et al [170] have designed the NASSS (nonadoption, abandonment, scale-up, spread, and sustainability) framework. In the case of CCDS systems for early sepsis detection in hospitals, the 7 NASSS framework domains can be identified as follows: sepsis (the condition); the CCDS system (the technology); the commercial and health-associated value of CCDS systems (value proposition and value chain); the responding personnel (the adopters); the hospital setting (the organization); the local, state, or national health system (the wider system); and software plasticity (embedding and adaptation over time). Our findings demonstrate that these domains are extremely diverse across the included studies, presenting many variables and variable combinations, consequentially expanding the complexity involved in sepsis CCDS system implementation. As the complexity of a system has been associated with its capacity for successful and sustainable implementation [25], this heterogeneity could detrimentally impact the performance of sepsis CCDS systems. To counter this issue, Greenhalgh et al [25] highlights the importance of system usability and adaptability, suggesting that a user-centered and iterative approach is needed, centralizing the involvement of relevant providers in the implementation plan. Unfortunately, our findings indicate that few of the included studies investigated the usability of sepsis CCDS systems.

A large proportion of studies did not specify important study design, CCDS system, and main outcome details (Tables 1-3). An unexpectedly high number of journal articles did not report these details nor did most conference abstracts; however, this is more understandable given word limit constraints. Of particular concern is that almost two-thirds of the included journal articles were found to have an average or poor clarity of outcome reporting (Figure 4). None of the studies included in this review published the use of reporting guidelines, likely because of many journals not specifically requiring it. Overall, we found that the quality of reporting is low and identified a need for improved reporting and transparency throughout the literature.

This scoping review comprehensively canvassed the literature investigating knowledge-based implemented CCDS systems for early sepsis detection in adult hospital patients. Its strength lies in this breadth of coverage and the wide range of study elements examined. The review followed the PRISMA-ScR expansion [28] guidelines, the Joanna Briggs Institute Reviewer’s Manual [27], and the framework presented by Arksey and O’Malley [29].

A limitation of this scoping review is that it only included studies written in English or had English translations readily available. Furthermore, only a sample of the charted data was double-checked by a second reviewer, potentially resulting in a higher error margin. However, the data charting forms were well structured, and any issues occurring during charting were fully discussed among the research team to reach consensus.

This review highlights the extensive variability in the design, outcomes, and system characteristics in studies investigating the use of CCDS for the early detection of sepsis in adult inpatients. This heterogeneity can be largely attributed to the considerable complexity of sepsis, CCDS software, and the hospital environment. Our findings have identified clear gaps in the current literature, with few studies investigating CCDS system usability, cost, or the effects on patient morbidity. There are limited studies conducted outside the United States or with robust study designs. Our findings have illustrated frequent poor reporting of CCDS system information and study outcomes. It is critically important for future research to close these knowledge gaps, ensuring comprehensive evaluation of these rapidly emerging sepsis CCDS systems.