Background: Evidence-based infectious disease and intensive care management is more relevant than ever. Medical expertise in the two disciplines is often geographically limited to university institutions. In addition, the interconnection between inpatient and outpatient care is often insufficient (eg, no shared electronic health record and no digital transfer of patient findings).
Objective: This study aims to establish and evaluate a telemedical inpatient-outpatient network based on expert teleconsultations to increase treatment quality in intensive care medicine and infectious diseases.
Methods: We performed a multicenter, stepped-wedge cluster randomized trial (February 2017 to January 2020) to establish a telemedicine inpatient-outpatient network among university hospitals, hospitals, and outpatient physicians in North Rhine-Westphalia, Germany. Patients aged ≥18 years in the intensive care unit or consulting with a physician in the outpatient setting were eligible. We provided expert knowledge from intensivists and infectious disease specialists through advanced training courses and expert teleconsultations with 24/7/365 availability on demand respectively once per week to enhance treatment quality. The primary outcome was adherence to the 10 Choosing Wisely recommendations for infectious disease management. Guideline adherence was analyzed using binary logistic regression models.
Results: Overall, 159,424 patients (10,585 inpatients and 148,839 outpatients) from 17 hospitals and 103 outpatient physicians were included. There was a significant increase in guideline adherence in the management of Staphylococcus aureus infections (odds ratio [OR] 4.00, 95% CI 1.83-9.20; P<.001) and in sepsis management in critically ill patients (OR 6.82, 95% CI 1.27-56.61; P=.04). There was a statistically nonsignificant decrease in sepsis-related mortality from 29% (19/66) in the control group to 23.8% (50/210) in the intervention group. Furthermore, the extension of treatment with prophylactic antibiotics after surgery was significantly less likely (OR 9.37, 95% CI 1.52-111.47; P=.04). Patients treated by outpatient physicians, who were regularly participating in expert teleconsultations, were also more likely to be treated according to guideline recommendations regarding antibiotic therapy for uncomplicated upper respiratory tract infections (OR 1.34, 95% CI 1.16-1.56; P<.001) and asymptomatic bacteriuria (OR 9.31, 95% CI 3.79-25.94; P<.001). For the other recommendations, we found no significant effects, or we had too few observations to generate models. The key limitations of our study include selection effects due to the applied on-site triage of patients as well as the limited possibilities to control for secular effects.
Conclusions: Telemedicine facilitates a direct round-the-clock interaction over broad distances between intensivists or infectious disease experts and physicians who care for patients in hospitals without ready access to these experts. Expert teleconsultations increase guideline adherence and treatment quality in infectious disease and intensive care management, creating added value for critically ill patients.
Trial Registration: ClinicalTrials.gov NCT03137589; https://clinicaltrials.gov/ct2/show/NCT03137589
Worldwide, health workforce shortages are a pressing concern. By 2030, it is estimated that there will be a shortage of 9.9 million physicians and nurses worldwide [, ]. In addition, the proportion of older people in Europe will exceed 30% by 2050 [ , ]. Hence, health care systems must become more flexible and efficient, for example, through accelerated digitization. Infectious disease management, especially the management of sepsis, is one area in which the potential for digitization to reduce the global burden of disease is particularly strong.
Increasing antimicrobial resistance (AMR) poses a growing threat to patients. Important underlying drivers are the overuse and misuse of antibiotics. Every year, approximately 700,000 patients worldwide die from infections that are treatable with antibiotics . Global pandemics such as COVID-19 are likely to promote the overuse of antimicrobials, thereby facilitating the further development of AMR. AMR could cost US $100 trillion between now and 2050, with the annual mortality rate reaching 10 million over this period [ ].
In 2017, there were approximately 48.9 million cases of sepsis worldwide and 11 million sepsis-related deaths (19.7% of all deaths globally) . Sepsis is the most common cause of morbidity and mortality in intensive care units (ICUs) around the world. Although the sepsis-related mortality rate is continuously decreasing, it is still remarkably high (30%-50%) [ - ]. The associated costs are US $24 billion annually in the United States alone [ ].
Numerous studies have shown that adherence to clinical practice guidelines for antibiotic therapy and sepsis management is associated with improved patient outcomes [- ]. Alarmingly, compliance with these guidelines is low [ , - ]. International educational health care campaigns, such as the Choosing Wisely initiative, have responded to this global challenge by increasing professional awareness of evidence-based medicine [ ]. In general, the Choosing Wisely recommendations promote essential practices and the avoidance of unnecessary diagnostic, preventive, and therapeutic procedures [ , ].
Telemedicine has the potential to support these efforts. It facilitates direct, round-the-clock interactions between physicians who care for patients in hospitals with limited subspecialist staff and intensivists or infectious disease specialists located far away. Observational studies have demonstrated that expert teleconsultations can reduce sepsis-related mortality by approximately 25%, with a simultaneous increase in guideline adherence [, ]. Despite decades of intensive care research worldwide, no drug or other therapeutic measure has achieved a comparable reduction.
The aim of TELnet@NRW is to establish and evaluate a telemedical inpatient-outpatient network (24/7/365) to improve the application of evidence-based medicine in infectious disease management, especially the management of sepsis. We hypothesized that the establishment of a digital network based on expert teleconsultations increases treatment quality in inpatient and outpatient care for these 2 subspecialties.
Trial Design and Ethics Approval
TELnet@NRW was a multicenter, stepped-wedge cluster randomized trial conducted at 2 university hospitals (Aachen and Muenster), 17 hospitals and 103 outpatient physicians’ offices associated with 2 physician networks. The protocol is publicly available , and the trial was prospectively registered at ClinicalTrials.gov (NCT03137589). The Ethics Committee of the Medical Faculty of the RWTH Aachen approved the study (EK 068/17). The study was funded by the Innovation Fund of the Federal Joint Committee (February 2017 to January 2020, funding code 01 NVF16010). Independent researchers from the Department of Health Economics and Health Care Management at Bielefeld University conceptualized and performed the analyses. The reporting of this trial is in line with the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) checklist [ ].
Inpatient and Outpatient Participants
Inpatients, aged ≥18 years, who had Staphylococcus aureus bacteremia or required intensive care treatment and who provided written informed consent were eligible for inclusion.
Outpatients, aged ≥18 years, with a possible infectious presentation based on the International Classification of Primary Care, who provided written informed consent were eligible for enrollment. Measles vaccination rates in children were also evaluated.
Because of the complexity and diversity of possible diagnoses and the extremely high total number of patients during the entire study period, triage for study enrollment was carried out by the attending physician on-site.
We excluded minors and patients who did not formally consent to participate in the study. In addition, patients who were in a dependent or employment relationship with the sponsor or one of the investigators or the principal investigator and patients who lived in an institution as mandated by a legal or administrative order were excluded from the study.
Initially, the standardized technical requirements at each participating site were established in line with the relevant guideline recommendations . This also included setting up a wireless local area network in hospitals and medical practices. A secure, privacy-compliant infrastructure was used for communication, including 2 high-encryption audio-video conference systems and the certified data exchange platforms FallAkte Plus (Healthcare IT-Solutions) and ELVI (CompuGroup Medical), which complied with the General Data Protection Regulation (EU Regulation 2016/679). Overall, our data protection measures were externally reviewed by independent data protection experts and continuously monitored throughout the study. The video conferencing infrastructure met high requirements in terms of quality, data security and portability. This infrastructure operated in a high-security, closed network (hardware virtual private network) or on dedicated lines.
Interventions: Expert Teleconsultations Plus Advanced Training Courses
The innovative telemedical network TELnet@NRW involved outpatient-inpatient cooperation. Separate facilities (hospitals of different levels of care and outpatient physicians’ offices) established a new digital health care structure for North Rhine-Westphalia, Germany. TELnet@NRW provided expert knowledge from 2 university hospitals to participating hospitals and outpatient physicians through expert teleconsultations (24/7/365 availability). Expert teleconsultations were provided on request after the initial triage was carried out by the attending physician on-site. Consultants for intensive care medicine participated in key care processes 24/7, whereas infectious disease specialists were available once weekly and on demand, including participation in rounds, additional expert teleconsultations, emergency consultations, and audits of clinical patient data. Before implementation of the expert teleconsultations, participants received advanced training courses on guideline-compliant treatment.
Study Schedule and Data Collection
All clusters went through three different study phases: During the preintervention phase, pseudonymized patient data from routine care was documented (details are provided inand , with the preintervention phase shown in red). During the subsequent transition phase (shown in white), the required hardware infrastructure was set up at the different study sites. Then, participants received on-site training according to their cluster schedule. To familiarize participants with the new processes, expert teleconsultations were already provided during the transition phase. Data on the effects of the intervention were collected during the following intervention phase (shown in blue). Primary data were generated using standardized case reporting forms. For the analyses of influenza and measles vaccination rates, we used routine outpatient claims data from the Association of Statutory Health Insurance Physicians (AHIP).
The primary outcome measure was adherence to the 10 Choosing Wisely recommendations for infectious diseases provided by the German Society for Infectious Diseases, which are applicable to both inpatient and outpatient care; these contain 5 Dos and Don’ts for infectious disease management (for each definition, please refer to) [ ]. Notably, the first 2 Dos address important quality indicators in intensive care medicine as they are associated with lower mortality [ , , , ]. For improved overview, shows the 10 Choosing Wisely recommendations sorted by their applicability to the inpatient and outpatient sector.
Secondary outcome measures for the inpatient sector were rate of sepsis therapy in compliance with guidelines (in compliance with the Surviving Sepsis Campaign guidelines for the management of severe sepsis and septic shock, defined as adherence to the 3- and 6-hour sepsis bundles ); rate of acute respiratory distress syndrome (ARDS) therapy in compliance with guidelines (measured against the evident ventilation targets, ventilation with low ventilation volumes, and low peak pressures; with controlled ventilation; breath volume of 6 mL/kg calculated ideal body weight; positive end-expiratory pressure setting in proportion with the necessary FiO2; and plateau pressure <30 cm H2O [ ]); and ICU and sepsis-related mortality, hospital mortality, ICU length of stay (LOS) and hospital LOS, rate of patients with dialysis at discharge from the ICU, and rate of transfer transport (defined as rate of patients discharged to another hospital).
Secondary outcome measures for the inpatient sector were health-related quality of life (measured using the 36-Item Short Form Survey version 2.0 questionnaire).
In our protocol, the 3 process variables rate of sepsis diagnosis, rate of ARDS diagnosis, and rate of undiagnosed sepsis were listed as part of the secondary outcomes. We corrected this in the report. All process variables are now reported as such.
Primary outcome sorted by sector.
Primary outcome inpatient sector: adherence to the following Choosing Wisely recommendations
- “Staphylococcus aureus bloodstream infection imperatively needs efficacious antimicrobial treatment and identification and elimination of the source of infection” [P1]
- “In critically ill patients with signs of infection, early appropriate antibiotic therapy is crucial after obtaining cultures, and treatment should be regularly re-evaluated” [P2]
- “Prescribe oral forms of highly bioavailable antimicrobial agents to patients who can reliably receive and absorb medications via the enteral route” [P5]
- “Do not treat asymptomatic bacteriuria with antibiotics” [N2]
- “Do not treat Candida recovered from respiratory or gastrointestinal tract specimens” [N3]
- “Do not extend the administration of prophylactic antibiotics after surgery (after the patient has left the operating room)” [N4]
- “Do not treat elevated C‐reactive protein or procalcitonin levels in serum with antibiotics in patients not presenting signs or symptoms of infection” [N5]
Primary outcome outpatient sector: adherence to the following Choosing Wisely recommendations
- “Annual influenza vaccination should be given to individuals aged >60 years, patients with specific comorbidities, and people (eg, health care workers) who may infect vulnerable persons” [P3]
- “All children should receive the measles vaccine, and adults born after 1970 without prior documented vaccination against measles should get at least one dose of the vaccine” [P4]
- “Avoid prescribing antibiotics for uncomplicated upper respiratory tract infections including bronchitis” [N1]
- “Do not treat asymptomatic bacteriuria with antibiotics” [N2]
Randomization and Masking
Participating hospitals and outpatient physician offices were randomly assigned to 4 clusters of 4 to 5 hospitals and 4 clusters of 23 to 28 outpatient physicians’ offices (and ). Participants were randomly assigned to one of the clusters with different start dates for the intervention phase by an independent statistician using a computer-generated random allocation sequence. It was not possible to mask the health care staff or patients, as they were involved in the delivery of the intervention.
Primary outcomes were evaluated using binary logistic regression models (primary data) and zero-or-one inflated beta regression models (AHIP data).
In the analyses of the inpatient data, we controlled for the treating hospital, patient age, and the Sequential Organ Failure Assessment (SOFA) score at enrollment in the study . SOFA scores were calculated based on the data in routine clinical patient records. Missing baseline values for the different SOFA subscores were imputed, if available, by measurements within the first 3 days after enrollment (next observation carried backward) [ ]. Only patients with complete SOFA scores (ie, measurements for all 6 subscores) at baseline were included in the analyses.
To differentiate the training effect and the external effect of expert teleconsultations from the direct counseling effect, the intervention group was separated into patients with and without expert teleconsultations, as not all patients who received interventions were treated with expert teleconsultations. Models for ICU and sepsis-related mortality and sepsis bundle compliance were specified in the same manner. Effects on the LOS were estimated using linear, gamma, and log-linear regression models with the same control variables.
In the evaluation of the primary outpatient data, we controlled for the treating outpatient physician and patient age. Three different models were estimated for each outcome:
- Model 1 contained only the group variable (group) and the control variables.
- In addition, model 2 contained a count variable (n) that recorded the number of expert teleconsultations the outpatient physician had already used before the visit with the respective patient.
- Finally, model 3 contained a quadratic term of the count variable (n2) to map possible learning curves among outpatient physicians in the sense of a decreasing marginal utility of the expert teleconsultations.
In the AHIP data, influenza and measles vaccination rates were documented quarterly at the practitioner level. To isolate the effect of the intervention on the vaccination rates, we controlled for the treating practitioner, the number of patients during the quarter and seasonal or quarterly effects.
Baseline group differences were tested using odds ratios (ORs) for binary variables or 2-tailed t tests for metric variables. Data cleaning and statistical analyses were performed in R (R Foundation for Statistical Computing; version 3.6.3) using the functions glm for logistic models and gamlss for beta regression models. CIs for regression estimates were computed using the confint command from the stats package, which calculated the CIs based on profile likelihood estimation. All analyses were based on a significance level of α=.05. The model structures are detailed in the.
A total of 17 hospitals and 103 outpatient physicians underwent randomization. The participating hospitals had between 101 and 449 beds; participating ICUs had on average 10 ICU beds (range 5-14) and were mixed medical-surgical ICUs staffed with anesthesiologists and internists. Hospitals were in both urban and rural areas. Of the 17 hospitals, 8 (47%) served populations <50,000; 7 (41%) served populations from 50,000 to 100,000; and 2 (12%) served populations >250,000. Most outpatient physicians were in urban areas with a high outpatient physician density per capita. Among the participating doctors, multiple specialties were represented, and the majority were general practitioners, internists, ophthalmologists, or gynecologists. Between May 3, 2017, and September 30, 2019, we enrolled patients (n=159,424) who required infectious disease or intensive care treatment in our study (). Overall, we provided 8505 inpatient expert teleconsultations. For outpatients, the average teleconsultation rate was 1.33% (1980/148,839). In the following, we report first the results of the inpatient sector and then the results of the outpatient sector.
Inpatient Study Enrollment
After initial triage by the attending physicians, we enrolled 10,585 inpatients (). The baseline characteristics of the inpatients are detailed in . A total of 0.57% (60/10,585) of the patients with incomplete data were excluded from the analysis, and we provide details on the distribution of missing values before and after imputation ( ). For the recommendation Staphylococcus aureusbloodstream infection imperatively needs efficacious antimicrobial treatment and identification and elimination of the source of infection, the SOFA score was not included in the statistical model, as we mainly observed non-ICU patients for this outcome, for whom the relevant parameters to calculate the score are not routinely recorded.
Compared with those in the control group, inpatients in the intervention group were older (mean age 69.25 years in the control group vs 72.14 years in the intervention group) and had higher SOFA scores at baseline (mean SOFA 3.58 in the control group vs 4.12 in the intervention group). The higher morbidity in the intervention group also manifested itself in a higher sepsis rate (5% vs 9%) and a higher ARDS rate (13% vs 17.9%). For the outcome-specific analysis samples, we found no fundamental deviations from the characteristics of the overall sample.
|Inpatient characteristics||Control group||Transition group||Intervention group||Intervention group versus control group|
|Difference (95% CI)||P value|
|Patients, mean (SD)||4099||3575||2851||N/Aa||N/A|
|Age (years), mean (SD)||69.25||70.34||72.14||2.89 (2.182-3.591)||<.001|
|Sex, n (%)|
|Male||1920 (46.8)||1616 (45.2)||1415 (49.6)||N/A||.97|
|Female||2177 (53.1)||1958 (54.8)||1430 (50.2)||N/A||.97|
|Other||2 (0)||1 (0)||6 (0.2)||N/A||N/A|
|SOFAb score at baseline, mean (SD)||3.58||3.72||4.12||0.54 (0.408-0.687)||<.001|
|Sepsis incidence, n (%)||206 (5)||286 (8)||256 (9)||N/A||<.001|
|ARDSc incidence, n (%)||531 (13)||696 (19.5)||511 (17.9)||N/A||<.001|
aN/A: not applicable.
bSOFA: Sequential Organ Failure Assessment.
cARDS: acute respiratory distress syndrome.
Inpatient Primary Outcomes
We found significant between-group differences in the management of Staphylococcus aureus bloodstream infections (Staphylococcus aureus bloodstream infection imperatively needs efficacious antimicrobial treatment and identification and elimination of the source of infection, P1). As expert teleconsultations were provided to all patients in this analysis, we did not divide the intervention group to estimate the effects of the intervention on this recommendation. Patients in the intervention group were significantly more likely to be treated in accordance with the recommendation (OR 4.004, 95% CI 1.828-9.202; P=.001;).
The direct effect of expert teleconsultations became also evident in the treatment of critically ill patients with severe sepsis and septic shock. We found significant between-group differences for the recommendation In critically ill patients with signs of infection, early appropriate antibiotic therapy is crucial after obtaining cultures, and treatment should be regularly re-evaluated (OR 6.822, 95% CI 1.271-56.607; P=.04;). However, patients in the intervention group who did not receive expert teleconsultations also received treatment that was more in line with the guideline recommendation than that received by the control group. Notably, across all included patients diagnosed with severe sepsis and septic shock, adherence to this recommendation was negatively associated with patient age ( ).
We found no significant intervention effects for the recommendation Prescribe oral forms of highly bioavailable antimicrobial agents to patients who can reliably receive and absorb medications via the enteral route ().
Regarding the recommendations Do not treat asymptomatic bacteriuria with antibiotics (n=24) and Do not treat Candida recovered from respiratory or gastrointestinal tract specimens (n=32), we had too few observations to generate logistic regression models ().
Regarding the extension of the period of treatment with prophylactic antibiotics after surgery once a patient has left the operating room, we observed a higher guideline adherence in patients in the intervention group who did not receive expert teleconsultations than in patients in the control group (OR 9.372, 95% CI 1.519-111.467; P=.04;). However, the estimated coefficient for the portion of the intervention group who directly received expert teleconsultations remained statistically nonsignificant.
Furthermore, no significant intervention effects could be found for the recommendation Do not treat elevated C‐reactive protein or procalcitonin levels in serum with antibiotics in patients not presenting signs or symptoms of infection (). It should be noted that the latter was already fulfilled in 90% (531/590) of the control cases.
|P1b (N=186)||P2c (N=211)||P5d (N=126)||N4e (N=193)||N5f (N=919)|
|Compliance, % (n/N)||ORg (95% CI)||P value||Compliance, % (n/N)||OR (95% CI)||P value||Compliance, % (n/N)||OR (95% CI)||P value||Compliance, % (n/N)||OR (95% CI)||P value||Compliance, % (n/N)||OR (95% CI)||P value|
|SOFAh score||—i||—||—||N/Aj||0.973 (0.863-1.096)||.65||N/A||1.355 (1.064-1.787)||.02||N/A||1.164 (0.753-1.879)||.51||N/A||0.772 (0.608-0.975)||.03|
|Age||N/A||0.973 (0.944-1.000)||.06||N/A||0.952 (0.914-0.987)||.01||N/A||0.995 (0.953-1.042)||.82||N/A||1.048 (1.008-1.093)||.02||N/A||0.993 (0.979-1.007)||.34|
|Control group||16.3 (15/92)||Refk||N/A||49.2 (29/59)||Ref||N/A||21.6 (19/88)||Ref||N/A||85.3 (110/129)||Ref||N/A||90.0 (531/590)||Ref||N/A|
|Intervention group||45.7 (43/94)||4.004 (1.828-9.202)||<.001||—||—||—||—||—||—||—||—||—||—||—||—|
|Without teleconsultation||—||—||—||81.8 (108/132)||4.718 (2.032-11.563)||<.001||0.00 (0/14)||0.000 (0.000, 1.032)||.99||93.9 (31/33)||9.372 (1.519-111.467)||.04||84.9 (163/192)||0.990 (0.542-1.834)||.97|
|With teleconsultation||—||—||—||90.0 (18/20)||6.822 (1.271-56.607)||.04||25.0 (6/24)||1.135 (0.179, 7.493)||.89||80.6 (25/31)||1.744 (0.326, 12.861)||.54||92.1 (125/137)||1.463 (0.666-3.416)||.36|
aEach model also controlled for hospital specific effects, which are not reported individually in this table; CIs were calculated based on profile likelihood estimation.
bPrimary outcome P1: Imperatively start antimicrobial treatment and remove the focus on Staphylococcus aureus bloodstream infection.
cPrimary outcome P2: Critically ill patients with signs of infection need early appropriate antibiotic therapy.
dPrimary outcome P5: Prefer oral formulations of highly bioavailable antimicrobials whenever possible.
ePrimary outcome N4: Do not prolong prophylactic administration of antibiotics in patients after they have left the operating room.
fPrimary outcome N5: Do not treat an elevated C‐reactive protein or procalcitonin level with antibiotics in patients without signs of infection.
gOR: odds ratio.
hSOFA: Sequential Organ Failure Assessment.
iDue to differences model specifications, the respective variables were not included in all models.
jN/A: not applicable.
kRef: reference group.
Inpatient Secondary Outcomes
Significant between-group differences were also found in the adherence to the 3- and 6-hour sepsis bundles for patients with the diagnoses of severe sepsis and septic shock (and ). Overall (0-6 hours), patients in the intervention group with expert teleconsultations outperformed the controls with regard to sepsis bundle compliance (OR 7.739, 95% CI 2.379-28.026; P=.001). This was mainly driven by an improvement in compliance with the 4 to 6-hour bundle. With expert teleconsultations, the odds of being treated in accordance with the guideline recommendations in the 4- to 6-hour therapy course after receiving a diagnosis of sepsis were 14.2 times higher (OR 14.245, 95% CI 3.121-85.424; P=.001) than in the control group. Overall and for the 3- and 6-hour sepsis bundles, we found significant improvements in compliance for patients in the intervention group who did not receive direct telemedical treatment support.
Regarding treatment quality in patients with mild ARDS, the descriptive analysis showed an increase in compliance from 7.4% (16/217) in the control group to 18.4% (9/49) in the intervention group without expert teleconsultations and 11.8% (19/161) in the group with expert teleconsultations. In the logistic regression model, a significant OR of 3.621 (95% CI 1.256-10.319; P=.02) was obtained for patients without expert teleconsultations. Patients with expert teleconsultations also showed a significantly increased chance of correct treatment mild ARDS (OR 2.355, 95% CI 1.023-5.516; P=.04). For patients with moderate or severe ARDS, we could not demonstrate these effects. A detailed description of this outcome is presented in.
Regarding ICU mortality (OR 1.276, 95% CI 0.909-1.794; P=.16) and sepsis-related mortality (OR 0.680, 95% CI 0.230-10.874; P=.37), no statistically significant intervention effects were found in our model estimations; however, a reduction in the sepsis-related mortality rate of 5% was achieved (19/66, 28.8% in the control group vs 50/210, 23.8% in the intervention group). Also, no significant improvements with regard to hospital mortality were observed.
ICU LOS was significantly longer for intervention patients with (+1.971 days, 95% CI 1.858-27.708; P=.004) and without (+2.253 days, 95% CI 2.235-40.535; P=.002) expert teleconsultations than for the respective controls. Regarding hospital LOS, it is noticeable that patients in the control group were hospitalized longer (mean 16.3 days, 95% CI 15.65-16.97 days) than patients in the intervention group without expert teleconsultations (mean 14.15 days, 95% CI 12.96-15.35 days) but for shorter periods than patients with expert teleconsultations (mean 20.62 days, 95% CI 19.55-21.70 days). The linear regression model confirms this result for patients with expert teleconsultations. These patients stayed on average 4.6 days (β=4.610 days, 95% CI 3.316-5.905 days; P<.001) longer in hospital than control patients. For the group of patients without expert teleconsultations, there was no significant difference to the control group.
A total of 0.3% (6/1983) of the patients in the control group and 1.1% (8/739) in the intervention group were discharged from hospital on dialysis. With this very low prevalence, effects of any intervention cannot be shown.
Overall, 4.38% (86/1965) of the patients in the control group were transferred to another hospital during our study. In the intervention group, this proportion was 11.34% (143/1261). Patients with expert teleconsultations were transferred more frequently (101/857, 11.8%) than patients in the intervention group without expert teleconsultations (42/404, 10.4%). The model calculation also shows a significant intervention effect. Patients who received an expert teleconsultation had a 2.9-fold higher chance of being transferred (OR 2.903, 95% CI 2.012-4.186; P<.001). In patients in the intervention group without expert teleconsultations, this chance was also significantly increased compared with that in the control group (OR 2.432, 95% CI 1.570-3.721). Overall, the analyses thus show an increase in the number of transfers owing to the intervention.
Outpatient Study Enrollment
In the outpatient sector, 148,839 patients were enrolled in our study. The intervention group differed significantly from the control group with regard to their distribution between the 2 physician networks. Baseline characteristics are detailed in, and provides details concerning study enrollment.
|Outpatient characteristics||Control group||Transition group||Intervention group||Intervention group versus control group|
|Difference (95% CI)||P value|
|Patients, mean (SD)||47,250||67,650||33,939||N/Aa||N/A|
|Age (years), mean (SD)||42.08||40.50||42.20||0.12 (−0.4538 to 0.2208)||.49|
|Sex, n (%)|
|Male||25,908 (54.8)||36,962 (54.6)||18,584 (54.8)||N/A||.83|
|Female||21,342 (45.2)||30,688 (45.4)||15,355 (45.2)||N/A||.83|
|Physician network, n (%)|
|MuMb||31,248 (66.1)||43,412 (64.2)||21,388 (63)||N/A||<.001|
|GKSc||16,002 (33.9)||24,238 (35.8)||12,551 (37)||N/A||<.001|
aN/A: not applicable.
bMuM: Medizin und Mehr eG.
cGKS: Gesundheitsnetz Köln-Süd eV.
Outpatient Primary Outcomes
Use of expert teleconsultation was associated with a significantly higher degree of adherence to the guideline recommendations for antibiotic therapy. Overall, the treatment of patients in the intervention group was significantly more compliant than that in the control group with regard to the Choosing Wisely recommendation for the treatment of uncomplicated upper respiratory tract infections (N1; OR 1.343, 95% CI 1.155-1.562; P=.001; model 1). This effect was significantly influenced by the number of expert teleconsultations conducted, as indicated by the estimated coefficient of the count variable (OR 1.007, 95% CI 1.001-1.013; P=.04; model 2). In addition, the estimated coefficient for the quadratic term of the count variable showed a statistically significant negative effect, which illustrates the decreasing marginal utility of the expert teleconsultations (OR 0.9998, 95% CI 0.9996-0.9999; P=.001; model 3). Here, we also observed an accumulation of noncompliance in the treatment of older patients ().
Our telemedical inpatient-outpatient network also achieved significant results with regard to the management of asymptomatic bacteriuria (N2). Patients in the intervention group were more likely to be treated in line with the guideline recommendations than were the controls (OR 9.312, 95% CI 3.794-25.936; P<.001; model 1). This effect was influenced more by the number of expert teleconsultations conducted than by the training of the treating physicians (OR 1.533, 95% CI 1.212-2.190; P=.004; model 2). Although not statistically significant, we also observed a trend toward a decreasing marginal utility of the expert teleconsultations with regard to this outcome (model 3). The Choosing Wisely recommendations also provide advice for increasing influenza (P3) and measles vaccinations (P4). For these analyses, we examined the quarterly vaccination rates at the physician level during the study period. In our basic model, we found no significant effect of expert teleconsultations on influenza vaccinations (rate ratio 1.089, 95% CI 0.911-1.302; P=.34). However, to better capture seasonal effects on the vaccination rates, we additionally constructed a model in which we extended the observation period and considered the transition phase as part of the intervention phase because the expert training occurred at the end of the control phase. In this model, intervention physicians had significantly higher influenza vaccination rates (rate ratio 1.204, 95% CI 1.079-1.344; P=.001). Regarding measles vaccination rates, we found no significant intervention effects in either model.
|Compliance, % (n/N)||Model 1||Model 2||Model 3|
|ORb (95% CI)||P value||OR (95% CI)||P value||OR (95% CI)||P value|
|Age||N/Ad||0.978 (0.975-0.980)||<.001||0.978 (0.975-0.980)||<.001||0.978 (0.975-0.980)||<.001|
|Control group||80.4 (7606/9456)||Refe||N/A||Ref||N/A||Ref||N/A|
|Intervention group||90.2 (5643/6258)||1.343 (1.155-1.562)||<.001||1.198 (0.997-1.438)||.05||0.999 (0.806-1.238)||.99|
|Number of teleconsultations||—f||—||—||1.007 (1.001-1.013)||.03||1.032 (1.015-1.049)||<.001|
|Squared number of teleconsultations||—||—||—||—||—||0.9998 (0.9996-0.9999)||.001|
|Age||N/A||0.996 (0.983-1.010)||.55||0.999 (0.985-1.012)||.83||0.999 (0.985-1.012)||.84|
|Control group||54.5 (145/266)||Ref||N/A||Ref||N/A||Ref||N/A|
|Intervention group||75.9 (369/486)||9.312 (3.794-25.936)||<.001||0.147 (0.010-1.218)||.11||0.092 (0.002-2.639)||.16|
|Number of teleconsultations||—||—||—||1.533 (1.212-2.190)||.004||1.717 (0.819-3.174)||.08|
|Squared number of teleconsultations||—||—||—||—||—||0.994 (0.978-1.038)||.65|
aEach model also controlled for physician-specific effects, which are not reported individually in this table; CIs were calculated based on profile likelihood estimation.
bOR: odds ratio.
cPrimary outcome N1: Avoid prescribing antibiotics for uncomplicated upper respiratory tract infections.
dN/A: not applicable.
eRef reference group.
fDue to the different model specification, the respective variables were not included in all models.
gPrimary outcome N2: Do not treat asymptomatic bacteriuria with antibiotics.
Outpatient Secondary Outcome
We had planned to assess health-related quality of life in outpatients measured with the 36-Item Short Form Survey version 2.0 questionnaire. Completed questionnaires from a total of 540 patients were available for the initial survey time t0 (study enrollment). This corresponds to 0.4% (540/148,839) of the patients included in the outpatient study. Of the 540 patients, 72 (13.3%) were eligible for a further survey after 3 or 12 months, as the contact data required for a renewed contact were collected for them at t0. The response rates in the follow-ups for these 72 patients were 32% (n=23) for t1 and 28% (n=20) for t2. An analysis of changes over time did not appear to be appropriate based on the low response rates for t1 and t2, and no analysis was performed.
Summary of Main Findings
To the best of our knowledge, TELnet@NRW is the largest telemedical cluster randomized controlled study in Europe, with more than 150,000 patients. We established a telemedical inpatient-outpatient network as a novel digital structure in the health care system, and we found that it measurably improved the quality of patient care. The consistent introduction and implementation of standardized communication using a certified electronic patient record was a feature that was essential for increasing the effectiveness of the processes involving the new digital health network. The key findings of this study suggest that expert teleconsultation is an effective tool to provide inpatient and outpatient physicians with evidence-based expertise on a large scale, thus improving guideline compliance and the quality of infectious disease and intensive care management.
Concerning the inpatient primary outcomes, our telemedical intervention had significant quality-improving effects on the management of Staphylococcus aureus bloodstream infections (P1), severe sepsis and septic shock (P2), and prophylactic antibiotic therapy (N4). Quality improvements for several outcomes reached not only those patients who were treated with direct expert teleconsultations but also other patients treated by the same physicians. This finding can be interpreted as a positive effect of the initial training courses and/or an indirect effect of the expert teleconsultations. It can therefore be assumed that the treating physicians also apply the knowledge acquired in teleconsultations to patients whose treatment is carried out without telemedical support.
In the outpatient sector, our telemedical intervention significantly increased guideline compliance in the management of uncomplicated upper respiratory tract infections (N1) and asymptomatic bacteriuria (N2). The chance of being treated according to the recommendations was positively associated with the number of teleconsultations already participated in by the outpatient physician before the respective patient visit. Furthermore, we found evidence for a decreasing marginal utility of these teleconsultations, which may reflect the physicians’ learning curve. To obtain a better understanding of physicians’ learning behavior and the associated practical implications, studies with longer observational periods are needed. Basic analyses of influenza and measles vaccination rates uncovered no significant intervention effects. However, when considering the initial transition phase as part of the intervention phase, influenza vaccination in the intervention group increased significantly. This could be explained either by the better statistical control of seasonal effects (as more observation quarters were included) or by the fact that the effects were larger immediately after the start of the intervention. The latter would indicate that the transition period was probably too long to capture the full potential of our intervention. Nevertheless, as this analysis deviates from the original study plan, it can only be interpreted as exploratory.
With regard to our secondary outcomes analyzed for the inpatient sector, we found that the provision of expert teleconsultations led to a higher overall sepsis bundle adherence, which was mainly driven by improvements in compliance with the 4- to 6-hour bundle. However, although the direct relationship between sepsis bundle compliance as a quality-of-care indicator and mortality is well documented in the scientific literature [- , , ], we did not observe significant intervention effects on ICU mortality, sepsis-related mortality, or hospital mortality.
Although its cross-sectoral applicability was an argument for selecting compliance with the Choosing Wisely recommendations as the primary outcome of this study, parts of these recommendations are not applicable to both inpatient and outpatient care, especially the ICU setting. Because gastrointestinal function is very often impaired in patients in the ICU and therefore the absorption of oral medications cannot be guaranteed, most medication is administered intravenously. Likewise, obtaining tracheal secretions and microbiological tests is a rarity in the outpatient sector, as the therapeutic consequence outside of serious infections is very low. As most patients in the ICU have an indwelling urinary catheter and are sedated, the criterion asymptomatic cannot be evaluated in the context of bacteriuria. Notably, the compliance with abstaining from treating elevated C‐reactive protein or procalcitonin levels in the serum with antibiotics in patients without signs or symptoms of infection was already 90% (531/590) in the control phase. In summary, 40% (4/10) of Choosing Wisely recommendations for infectious disease management are not fully applicable to ICU care.
Within the real-world setting of our trial, it was at the discretion of the practitioners for which patients an expert teleconsultation was requested. Hence, attending physicians tended to include patients thought to be more ill during the intervention phase of the study, which is reflected by the higher SOFA scores, sepsis, and ARDS incidences for inpatients in the intervention phase compared with those in the control phase. We controlled for such selection effects by including relevant variables in our regression models, but we could not rule out the presence of unknown confounders for which there were no data available. Although our regression models were adjusted for the SOFA score, this may not have fully controlled for the differences in the baseline risks of morbidity and mortality between the study groups. It is unclear to what extent this problem has been aggravated by the frequent occurrence of missing values for the different SOFA subscores and the associated need for data imputation. It should be highlighted that TELnet@NRW was not designed or powered to detect differences in sepsis-related mortality because the primary outcome focused on quality indicators for infectious disease management (). Nevertheless, in the treatment of patients with sepsis, the early detection of sepsis followed by the early initiation of therapy conforming to the recommendations in the guidelines significantly improves the clinical outcomes [ , , , , ]. Hence, if expert teleconsultations continue to be part of routine health care, we expect that this will also have a positive effect on mortality in ICUs, as has been reported in similar trials in the past [ , , ]. The equivalent is well documented in the literature regarding adherence to evidence-based management of Staphylococcus aureus bacteremia. Guideline adherence significantly improves clinical outcomes and reduces mortality [ - ].
As Hemming et al  note, control for secular effects plays a crucial role in stepped-wedge trials. However, especially the need for an appropriately long transition phase to implement the intervention at the participating sites impeded an adequate mapping of time in our statistical models. To address this shortcoming, we estimated secular effects on our inpatient primary outcomes under consideration of transition phase data in a secondary analysis. Our models incorporate time in two ways: (1) the days since the beginning of the study (ie, the beginning of the control phase) and (2) the days since the beginning of the transition phase. Results of the estimations are displayed in the . For most of the primary outcomes, we observed no significant time effects (P5, N4, and N5). The estimation for P2 shows (1) a significant negative overall time effect and (2) a significant positive effect since the beginning of the implementation of the intervention. Thus, we conclude that our intervention was a major driver of the observed improvements and that our primary model without time variables may rather tend to underestimate the intervention effects. For outcome P1, however, the estimation shows significant time effects pointing in the opposite direction. It can be assumed that the negative time effect observed since the beginning of the transition phase is largely due to implementation difficulties in the participating normal wards. However, we cannot rule out that the intervention effects measured in the primary evaluations were overestimated owing to a possible time trend.
Overall, the ﬁndings of this study should be interpreted in the context of its limitations. The participating hospitals, ICUs, and outpatient physicians were not chosen at random; instead, the 17 sites and 103 outpatient physicians were self-selected based on their willingness to participate in a study to improve patient care. However, we chose a randomized stepped-wedge design to control for clinical characteristics, demographics, and setting to protect our findings against bias. Nevertheless, our results should be interpreted with the consideration of the possibility for selection bias due to the on-site triage of patients. We controlled for such selection effects by including relevant variables in our regression models, but we could not rule out the presence of unknown confounders for which there were no data available. We also acknowledge potential bias related to secular trends in care given the fact that the control and intervention clusters did not overlap in time. Furthermore, for some outcomes, the real effect size remained uncertain, which is reflected by the large CI. This holds true especially for sepsis bundle compliance. Nevertheless, there is sufficient certainty that the associated ORs were >1; thus, intervention effects existed.
Despite the mentioned limitations, TELnet@NRW robustly demonstrated that a cross-sectoral health network, as a new digital structure in the health care system, can develop into a quality network that operates under the premise that cross-sectoral and interregional cooperation can significantly improve evidence-based care. On the basis of the technical equipment, the principles of TELnet@NRW are transferable from intensive care medicine and infectious disease management to other subspecialist medical fields that mostly rely on expert knowledge. Thus, the concept of TELnet@NRW can be adapted to other patient populations, other conditions, or other areas, in which expertise rather than equipment needs to be transported over large distances. Our results must also be interpreted in light of the most recent SARS-CoV-2 pandemic; a digital inpatient-outpatient health network is well suited to meeting pressing challenges faced by health care systems, which we will have to address in the future (eg, staff shortages in health care sectors, lack of experts in the geographic area, and aging societies). However, further research is needed with regard to the long-term patient-relevant effects of our telemedical solution and its cost-effectiveness especially in less complex cases in the outpatient setting.
The authors thank all the patients who participated in the study and the clinical and research staff at all the participating sites, without whose assistance the study would never have been completed. The authors wish to thank all members of the TELnet@NRW study group, who are listed in. The authors also thank Kassenärztliche Vereinigung Nordrhein and Kassenärztliche Vereinigung Westfalen Lippe for their support of TELnet@NRW.
The funder of the study had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
GM, KS, SWL, RD, SD, DB, CJ, and AP contributed to the study concept and design. GM, KS, RD, CJ, JE, AG, CL, SWL, MH, FJ-K, SD, DB, HS-S, SR, JK, and AP contributed to data acquisition. WG, DG, SE, and AP conceptualized and performed the statistical analysis. GM, KS, RD, WG, DG, SE, SD, AP, and CB contributed to data interpretation. GM, SD, CB, DG, SE, and WG wrote the report, with critical revision for important intellectual content by all other coauthors. All authors read and approved the final manuscript. GM supervised the study.
The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Conflicts of Interest
GM is the coordinator of the S1 Guideline Telemedicine in Intensive Care Medicine and chairman of the German Society for Telemedicine (DGTelemed). In the past, he has given lectures for Philips. GM is cofounder of Clinomic GmbH. WG, SE, and DG report grants from the Federal Joint Committee (Gemeinsamer Bundesausschuss), during the conduct of the study. AP is a cofounder of Clinomic GmbH. The authors declare no other competing interests.
Study schedule using a stepped-wedge design (inpatient sector).DOCX File , 94 KB
Study schedule using a stepped-wedge design (outpatient sector).DOCX File , 97 KB
Analysis algorithms of the 10 Choosing Wisely recommendations.DOCX File , 30 KB
Model structures.DOCX File , 19 KB
Study enrollment over time (inpatient sector).DOCX File , 98 KB
Missing value distribution of baseline Sequential Organ Failure Assessment scores.DOCX File , 19 KB
Regression analyses of sepsis bundle compliance.DOCX File , 23 KB
Regression analysis of acute respiratory distress syndrome therapy compliance.DOCX File , 23 KB
Study enrolment over time (outpatient sector).DOCX File , 158 KB
Regression analyses of secular effects on inpatient primary outcomes.DOCX File , 19 KB
TELnet@NRW Study Group.DOCX File , 18 KB
CONSORT-EHEALTH Checklist (V 1.6.1).PDF File (Adobe PDF File), 304 KB
- Global Strategy on Human Resources for Health: Workforce 2030. Geneva, Switzerland: World Health Organization; 2016.
- Commission Staff Working document on an action plan for the EU health workforce. Publications Office of the European Union. URL: https://op.europa.eu/de/publication-detail/-/publication/402bad92-c66f-4d45-9984-70e199dfd312/language-en [accessed 2022-02-10]
- Strategic research agenda for robotics in Europe 2014-2020. SPARC The Partnership for Robotics in Europe. 2013. URL: https://www.eu-robotics.net/cms/upload/topic_groups/SRA2020_SPARC.pdf [accessed 2022-01-10]
- Population structure and ageing. Eurostat. 2021. URL: https://ec.europa.eu/eurostat/statistics-explained/index.php/Population_structure_and_ageing [accessed 2022-02-10]
- Tackling drug-resistent infections globally: final report and recommendations. Review on Antimicrobial Resistance. 2016. URL: https://amr-review.org/sites/default/files/160518_Final%20paper_with%20cover.pdf [accessed 2022-02-10]
- Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. Lancet 2020;395(10219):200-211 [FREE Full text] [CrossRef] [Medline]
- Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, International Forum of Acute Care Trialists. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am J Respir Crit Care Med 2016;193(3):259-272. [CrossRef] [Medline]
- SepNet Critical Care Trials Group. Incidence of severe sepsis and septic shock in German intensive care units: the prospective, multicentre INSEP study. Intensive Care Med 2016 Dec;42(12):1980-1989. [CrossRef] [Medline]
- Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med 2013;41(2):580-637. [CrossRef] [Medline]
- Levy MM, Rhodes A, Phillips GS, Townsend SR, Schorr CA, Beale R, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Intensive Care Med 2014;40(11):1623-1633. [CrossRef] [Medline]
- Rhodes A, Phillips G, Beale R, Cecconi M, Chiche JD, De Backer D, et al. The Surviving Sepsis Campaign bundles and outcome: results from the International Multicentre Prevalence Study on Sepsis (the IMPreSS study). Intensive Care Med 2015;41(9):1620-1628. [CrossRef] [Medline]
- Deisz R, Rademacher S, Gilger K, Jegen R, Sauerzapfe B, Fitzner C, et al. Additional telemedicine rounds as a successful performance-improvement strategy for sepsis management: observational multicenter study. J Med Internet Res 2019;21(1):e11161 [FREE Full text] [CrossRef] [Medline]
- Wathne JS, Harthug S, Kleppe LK, Blix HS, Nilsen RM, Charani E, et al. The association between adherence to national antibiotic guidelines and mortality, readmission and length of stay in hospital inpatients: results from a Norwegian multicentre, observational cohort study. Antimicrob Resist Infect Control 2019;8:63 [FREE Full text] [CrossRef] [Medline]
- Dylis A, Boureau AS, Coutant A, Batard E, Javaudin F, Berrut G, et al. Antibiotics prescription and guidelines adherence in elderly: impact of the comorbidities. BMC Geriatr 2019;19(1):291 [FREE Full text] [CrossRef] [Medline]
- Lehmann C, Berner R, Bogner JR, Cornely OA, de With K, Herold S, et al. The "Choosing Wisely" initiative in infectious diseases. Infection 2017;45(3):263-268. [CrossRef] [Medline]
- Leone M, Ragonnet B, Alonso S, Allaouchiche B, Constantin JM, Jaber S, AzuRéa Group. Variable compliance with clinical practice guidelines identified in a 1-day audit at 66 French adult intensive care units. Crit Care Med 2012;40(12):3189-3195. [CrossRef] [Medline]
- Brunkhorst FM, Engel C, Ragaller M, Welte T, Rossaint R, Gerlach H, German Sepsis Competence Network (SepNet). Practice and perception--a nationwide survey of therapy habits in sepsis. Crit Care Med 2008;36(10):2719-2725. [CrossRef] [Medline]
- Fact sheet antimicrobial resistance. World Health Organization. 2018. URL: https://www.who.int/en/news-room/fact-sheets/detail/antimicrobial-resistance [accessed 2022-02-10]
- Choosing wisely: a special report on the first five years. ABIM Foundation. 2010. URL: https://www.choosingwisely.org/wp-content/uploads/2017/10/Choosing-Wisely-at-Five.pdf [accessed 2022-01-21]
- Lilly CM, McLaughlin JM, Zhao H, Baker SP, Cody S, Irwin RS, UMass Memorial Critical Care Operations Group. A multicenter study of ICU telemedicine reengineering of adult critical care. Chest 2014;145(3):500-507. [CrossRef] [Medline]
- Telemedizinisches, intersektorales Netzwerk als neue digitale Gesundheitsstruktur zur messbaren Verbesserung der wohnortnahen Versorgung. Fakultät für Gesundheitswissenschaften, Universität Bielefeld. URL: https://www.telnet.nrw/wp-content/uploads/TELnet@NRW_Evaluationsdesign-Analyseplan.pdf [accessed 2022-01-21]
- Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
- Telemedicine in intensive care medicine. Association of the Scientific Medical Societies in Germany. 2021. URL: https://www.awmf.org/leitlinien/detail/ll/001-034.html [accessed 2022-01-21]
- Lilly CM, Cody S, Zhao H, Landry K, Baker SP, McIlwaine J, University of Massachusetts Memorial Critical Care Operations Group. Hospital mortality, length of stay, and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. JAMA 2011;305(21):2175-2183. [CrossRef] [Medline]
- Kaasch AJ, Barlow G, Edgeworth JD, Fowler Jr VG, Hellmich M, Hopkins S, ISAC‚ INSTINCT‚ SABG‚ UKCIRG‚ and Colleagues. Staphylococcus aureus bloodstream infection: a pooled analysis of five prospective, observational studies. J Infect 2014;68(3):242-251 [FREE Full text] [CrossRef] [Medline]
- S3-Leitlinie Invasive Beatmung und Einsatz extrakorporaler Verfahren bei akuter respiratorischer Insuffizienz. Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften. 2017. URL: https://www.awmf.org/uploads/tx_szleitlinien/001-021l_S3_Invasive_Beatmung_2017-12.pdf [accessed 2022-01-21]
- Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22(7):707-710. [CrossRef] [Medline]
- Kenward MG, Molenberghs G. Last observation carried forward: a crystal ball? J Biopharm Stat 2009;19(5):872-888. [CrossRef] [Medline]
- Ferrer R, Artigas A, Levy MM, Blanco J, González-Díaz G, Garnacho-Montero J, Edusepsis Study Group. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA 2008;299(19):2294-2303. [CrossRef] [Medline]
- Dellinger RP, Carlet JM, Masur H, Gerlach H, Calandra T, Cohen J, et al. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Intensive Care Med 2004;30(4):536-555. [CrossRef] [Medline]
- Goto M, Schweizer ML, Vaughan-Sarrazin MS, Perencevich EN, Livorsi DJ, Diekema DJ, et al. Association of evidence-based care processes with mortality in staphylococcus aureus bacteremia at veterans health administration hospitals, 2003-2014. JAMA Intern Med 2017;177(10):1489-1497 [FREE Full text] [CrossRef] [Medline]
- Pérez-Rodríguez MT, Sousa A, López-Cortés LE, Martínez-Lamas L, Val L, Baroja A, et al. Moving beyond unsolicited consultation: additional impact of a structured intervention on mortality in Staphylococcus aureus bacteraemia. J Antimicrob Chemother 2019;74(4):1101-1107. [CrossRef] [Medline]
- López-Cortés LE, Del Toro MD, Gálvez-Acebal J, Bereciartua-Bastarrica E, Fariñas MC, Sanz-Franco M, REIPI/SAB group. Impact of an evidence-based bundle intervention in the quality-of-care management and outcome of Staphylococcus aureus bacteremia. Clin Infect Dis 2013;57(9):1225-1233. [CrossRef] [Medline]
- Nagao M, Yamamoto M, Matsumura Y, Yokota I, Takakura S, Teramukai S, et al. Complete adherence to evidence-based quality-of-care indicators for Staphylococcus aureus bacteremia resulted in better prognosis. Infection 2017;45(1):83-91. [CrossRef] [Medline]
- van Hal SJ, Jensen SO, Vaska VL, Espedido BA, Paterson DL, Gosbell IB. Predictors of mortality in Staphylococcus aureus bacteremia. Clin Microbiol Rev 2012;25(2):362-386 [FREE Full text] [CrossRef] [Medline]
- Hemming K, Taljaard M, McKenzie JE, Hooper R, Copas A, Thompson JA, et al. Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration. BMJ 2018;363:k1614 [FREE Full text] [CrossRef] [Medline]
|AHIP: Association of Statutory Health Insurance Physicians|
|AMR: antimicrobial resistance|
|ARDS: acute respiratory distress syndrome|
|CONSORT-EHEALTH: Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth|
|ICU: intensive care unit|
|LOS: length of stay|
|OR: odds ratio|
|SOFA: Sequential Organ Failure Assessment|
Edited by A Mavragani; submitted 06.10.21; peer-reviewed by M Ohl; comments to author 16.11.21; revised version received 10.01.22; accepted 22.01.22; published 02.03.22Copyright
©Gernot Marx, Wolfgang Greiner, Christian Juhra, Svenja Elkenkamp, Daniel Gensorowsky, Sebastian Lemmen, Jan Englbrecht, Sandra Dohmen, Antje Gottschalk, Miriam Haverkamp, Annette Hempen, Christian Flügel-Bleienheuft, Daniela Bause, Henna Schulze-Steinen, Susanne Rademacher, Jennifer Kistermann, Stefan Hoch, Hans-Juergen Beckmann, Christian Lanckohr, Volker Lowitsch, Arne Peine, Fabian Juzek-Kuepper, Carina Benstoem, Kathrin Sperling, Robert Deisz. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 02.03.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.