Investment in digital health is advancing rapidly. In 2020, the total global funding for digital health was the highest recorded at US $26.5 billion [1]. A global appetite for digital health, fueled recently by the COVID-19 pandemic and the associated rapid adoption of point-of-care technological solutions [2], including telehealth [3], has driven the digital disruption of health care. A core pillar of digital health investment is the digital transformation of hospitals [4].

The World Health Organization Global Strategy on Digital Health (2020-2025) recommends the implementation of a national digital health architecture, including digital hospitals [5]. Digital hospitals represent significant jurisdictional investments to improve the quality and safety of acute care [6]. A digital hospital uses a comprehensive electronic medical record (EMR) to achieve its clinical goals [7] and is becoming the predominant method of care delivery worldwide. These new digital hospital environments radically disrupt well-rehearsed clinical workflows and create unfamiliar environments for patients and clinicians, potentially affecting quality, safety, and experience of care [8-10].

It has been difficult to determine the value of digital hospital implementations as what is considered valuable changes over time and place and from person to person [11]. Previous studies evaluating digital health implementations have focused on three domains: (1) improving patient or hospital outcomes using quantitative evaluation [12]; (2) exploring patient [13] and clinician behavior, workflows, and attitudes toward EMRs or digital hospital transformations [10]; and (3) quantifying value using economic evaluations [4]. Evidence to date demonstrates conflicting impacts of EMRs on hospital practice, with positive indications for medication safety, guideline adherence, and decision support [12,14] and negative indications for physician-patient communication, staff attitude, and workflow disruption [15,16]. Focusing on the narrow aspects of digital health implementations has resulted in patchy assessments of the value of digital health technologies.

The Quadruple Aim is the overarching goal of a learning health care system of enhancing patient experience, improving population health, reducing health care costs, and improving the provider experience (Figure 1) [17]. The Quadruple Aim of health care has been proposed as a strategic compass to guide digital health investment planning, decision-making, and evaluation [11] and has been used in the health care workforce [18,19], innovation implementation [20], and COVID-19 pandemic [21] contexts to identify current trends and research gaps.

The experience of patients and clinicians has yet to be explored as important contributors to the Quadruple Aim. Previous evaluations of clinician experience have focused on individual retrospective recalls of attitudes, perceptions of EMR implementation [10], and observational time and motion studies [22]. Existing patient experience research has focused on bespoke digital systems for patient use (eg, internet-based care technologies, web-based patient platforms, and mobile health apps) or emerging trends (eg, COVID-19 impacts, effects of specific technologies, research methods, and new technologies) [23,24]. Traditional evaluations of technology in health care are selective in the outcomes they measure, with an overwhelming focus on clinical outcomes and efficiency.

To address this research gap, our research question was as follows: what is the clinician and patient experience of digital hospitals? We hypothesized that clinicians and patients would report digital hospital experiences positively (eg, patient safety benefits because of digital safeguards [25]), negatively (eg, productivity loss because of documentation burden [15]), and ambivalently (eg, no observed impact of the digital environment on patient experience [10]). The study aim was to conduct a systematic literature review and qualitative evidence synthesis to assess the clinician and patient experience of digital hospitals.

This review adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement checklist [26] (Multimedia Appendix 1 [26]) and the ENTREQ (Enhancing the Transparency in Reporting the Synthesis of Qualitative Research) guidelines [27] (Multimedia Appendix 2). The protocol for this review was registered in PROSPERO (CRD42021258719).

The PubMed, Embase, Scopus, CINAHL, and PsycINFO databases were searched twice—on June 24, 2021 (version [V1]), and subsequently on June 23, 2022 (version [V2])—using the same search strategy restricted by year of publication (2010 to the present) because of the relative novelty of the digital transformation of acute care.

The search strategy was initially developed in PubMed using the population, intervention, comparison, and outcome structure (Table 1) [28] and translated to other databases (Multimedia Appendix 3). We define experience as an individual’s perception of events, incorporating themes of expectation and satisfaction. Thus, synonyms for “experience” were used to characterize the outcome. A combination of indexed terms (eg, Medical Subject Headings) and keywords identified after consultation with a research librarian and subject matter experts was used. Truncations, synonyms, and terminological variations (eg, “EMR” vs “EHR”) were also used.

In total, 2 reviewers (YM and OJC) performed title and abstract screening. Full-text review was then performed based on the eligibility criteria. Backward citation tracking (snowballing) was used to identify additional articles in the reference lists of included articles and relevant reviews. Decision conflicts were resolved through internal discussion or by involving a third reviewer’s opinion (C Slade) when required.

Studies were included if they described a quantitative, qualitative, or mixed methods investigation of clinician or adult inpatient “experience” in a digital hospital (Table 2). This review focused on multidisciplinary clinicians and adult inpatients as a first step in synthesizing evidence of digital hospital experience. Pediatric inpatients were excluded because of possible environmental factors that could confound the digital hospital experience (eg, patient entertainment systems). Our study setting prioritized EMRs—a real-time patient health record that collects, stores, and displays clinical information in a tertiary setting [29] (eg, Cerner Millenium and Epic)—as the foundation of a digital hospital [30] as opposed to an electronic health record (or personal health record), which displays summarized patient information to the consumer in the community and across multiple health care providers [29] (eg, My Health Record [Australia], the National Health Service app [United Kingdom], and personal health record [Ministry of National Guard Health Affairs, Kingdom of Saudi Arabia]). The terms “EMR” and “EHR” may be used interchangeably in some countries, so both terms were adopted in the search strategy. The stage of digital hospital implementation (eg, EMR maturity [31]) was not considered.

The Covidence software (Veritas Health Innovation) and Excel (Microsoft Corp) facilitated data extraction of study details (Multimedia Appendix 4). For studies eligible for the qualitative evidence synthesis (eg, presence of verbatim participant quotes), additional data that explained the qualitative findings were extracted, including the primary and main themes; secondary and subthemes; minor and unexpected themes; participant quotations; and any text labeled within the “results” or “findings” (ie, narrative) sections, including data-driven discoveries, judgments, or explanations the researchers offered about their phenomena [32,33]. All extracted data were cross-checked by a second reviewer for accuracy, and any discrepancies were resolved through discussion. Owing to the heterogeneity in study design, populations, and outcome measures for quantitative studies, a meta-analysis was inappropriate, and a narrative synthesis of quantitative study results was conducted.

The quality of the included studies was assessed using the Mixed Methods Appraisal Tool (MMAT) [34]. Each study’s methodology was evaluated against 5 criteria (Yes, No, or Can’t tell) that differed between study designs (qualitative, quantitative randomized controlled trial, quantitative nonrandomized, quantitative descriptive, and mixed methods). The included studies were divided equally among 5 reviewers (LW, YM, C Slade, JK, and OJC). An MMAT star rating was generated for each article (Yes=1 star; up to 5 stars in total), a method adapted from a recent systematic review by Freire et al [35], and the scores were cross-checked by a second reviewer. Discrepancies were discussed and resolved within the research team. No study was excluded from the review based on its MMAT score.

Data synthesis was conducted in 2 stages in accordance with our study aims.

In total, 2059 studies were identified from the first search (V1), and an additional 462 studies were identified from the second search (V2; Figure 2). Following duplicate removal and title and abstract screening, a total of 109 studies (V1: n=84, 77.1%; V2: n=25, 22.9%) remained and underwent full-text review. Of these 109 studies, 61 (56%) met our inclusion criteria and were included in this review, comprising quantitative (n=39, 64%), qualitative (n=15, 25%), and mixed methods (n=7, 11%) designs.

In total, 52% (32/61) of the included studies met all 5 MMAT quality criteria (Multimedia Appendix 5 [44-104]). An additional 20% (12/61) of the studies met 4 out of 5 of the quality criteria. Only 7% (4/61) of the studies met ≤2 of the 5 quality criteria, of which 75% (3/4) were mixed methods studies and 25% (1/4) had a quantitative descriptive design. For these studies, a score of ≤2 indicated inadequate sampling, and in the case of the mixed methods studies, the integration of and inconsistencies between quantitative and qualitative elements were not adequately described.

A total of 18 studies (n=14, 78% qualitative and n=4, 22% mixed methods) with qualitative components were included in the qualitative evidence synthesis. Only 7% (1/15) of the qualitative studies in our review explored the patient experience in a digital hospital [52]; however, this study was excluded from the qualitative evidence synthesis and is reported narratively in the following paragraph. A total of 29% (2/7) of the mixed methods studies were also excluded for lacking direct participant quotes as per the exclusion criteria [50,75]; however, the quantitative results are reported in the previous sections.

Strauss [52] interviewed 11 patients about the dynamics with their nurses and the EMR. Similar to the qualitative evidence synthesis, participants described a positive perception of the EMR when the nurses acknowledged the participants before using the electronic device; however, many “expressed concerns [for] the privacy of their health record information.” Interestingly, participants’ expectations of the “clinical knowledge and competency of the nurse, within the technological arena, have increased with the implementation of the [EMR].”

Our findings revealed mixed and complex clinician and patient experiences of digital hospitals (Figure 3). Generally, clinicians reported positive overall satisfaction (17/24, 71% of the studies) with digital hospitals in quantitative measures; however, there were negative experiences for clinicians reported in qualitative studies, including compromised clinician-patient interactions, inefficient data workflows, and patient data privacy concerns. For example, acceptance of digital hospitals fluctuated over time and trended negatively if grounded in individual perceptions and beliefs (ie, “work as imagined”) yet trended positively if based on objective measures of practice (ie, “work as done”) [105]. These inconsistencies are likely reflective of the various contextual factors that influence experience, such as intervention design or stage of implementation. It is likely that the quantitative finding of mixed usability of EMRs explains clinician reliance on hybrid (digital and paper) workflows revealed in the qualitative evidence synthesis as clinicians seek to maintain their clinical workflow standard and validate data using additional sources of truth, such as paper [10].

The effect of digital hospitals on clinician-reported patient safety and clinician ability to deliver care were mixed; there was acknowledgment that digitization primarily reduces medication error risk but creates new risks driven by questionable data quality. In the small proportion of studies that explored patient experience (10/61, 16%), there was weak evidence supporting a positive association between digital hospitals and patient satisfaction scores. To our knowledge, this is the first review to systematically evaluate clinician and patient experience in digital hospitals and use qualitative evidence synthesis with machine learning (Leximancer) to consolidate identified themes in previous qualitative research into an empirical “umbrella” view of digital hospital experience.

Previous studies have evaluated the adoption of health ITs and identified enablers of and barriers to the routine use of EMRs in practice. The perceived value of the EMR to clinical workflows and data accessibility are key adoption facilitators, whereas cost and time consumption are barriers to adoption [106]. Our findings revealed that clinicians reported high satisfaction with digital hospitals and positively viewed data accessibility in quantitative measures; however, our qualitative evidence synthesis revealed themes of frustration with slow digital workflows and inconsistent data discoverability. Evidence of EMR adoption in low-income countries also highlights clinician perception of the EMR as a key facilitator and interoperability and clinician burnout as barriers, similar to our findings on the impact of EMR on workload and burnout symptoms [107].

Our review found that the patient experience of digital hospitals was reported disproportionately less frequently than the clinician experience. Evidence of patient satisfaction with EMRs was systematically reviewed in 2013, and it was found that a small number of studies (n=8) indicated positive patient satisfaction with the EMR in mixed settings across primary care, emergency, and outpatient departments [13]. This evidence is consistent with our findings of a positive or neutral association but remains grounded in cross-sectional methods that warrant rigorous trial evaluation. Beyond the clinician-facing EMR, patient-centered digital health records have emerged as a mechanism to engage and empower consumers living with chronic conditions. Patient portals within EMRs can contribute positively to health care quality and safety by improving medication adherence and clinician-patient communication [24] and have been shown to improve patient care navigation and disease knowledge without adverse effects and with high patient satisfaction [108]. Measuring satisfaction with digital hospitals using a simple quantitative scale is unlikely to capture the complexity and heterogeneity of digital hospital environments. There was dissonance in clinician perspective on data accessibility (or discoverability) between quantitative and qualitative studies. Clinicians reported objective satisfaction with data accessibility and positive attitudes toward data quality; however, they were dissatisfied with the inefficient workflows required to generate high-quality data (ie, input) and the ability to leverage these data for secondary use (ie, output).

Although patient experience is consistently positively associated with patient safety, clinical effectiveness, and self-rated and objectively measured health outcomes [109], there remains a paucity of empirical research that directly investigates patient experience in digital hospital environments. Our results revealed that, in a small number of studies, patient satisfaction scores were positively associated with digital hospitals (4/61, 7%) or remained unchanged (3/61, 5%). Patients and clinicians shared positive overall impressions of EMRs and negative attitudes toward data privacy in a digital hospital environment; however, this result must be considered in the disproportionate context of patient and clinician experience data reported in studies (10/61, 16% vs 55/61, 90%, respectively).

Patient experience in a digital hospital was sometimes inferred from surrogate measures, including hospital recommendations and discharge information quality, that do not capture the complexity of personal experience. For patients, crude quantitative measurement of satisfaction with digital hospitals neglects the complex nature of the digital patient experience [110] or emerging consumer digital health themes (eg, ethical implications, security, choice, privacy, transparency, accuracy, user-friendliness, and equity of access) [111,112]. The authors support the call by Viitanen et al [23] to develop a framework to describe the different aspects of patient experience and correlate them with appropriate methods for studying patient experience in this context.

The effect of digital hospitals on the clinician-patient relationship was consistently reported by clinicians as a negative outcome of digitization. These results align with the well-researched relationship between EMRs and the clinician-patient dynamic, with evidence supporting negative communication outcomes (eg, rapport, quality of interaction, and time) [113] from a clinician perspective. Patient perceptions of clinician-patient communication when using EMRs are relatively stable in previous systematic reviews despite objective studies describing potentially negative (eg, interrupted speech) and positive (eg, facilitating questions) effects [16]. Patient portals within EMRs can improve clinician-patient communication and should be considered a necessary infrastructure for health services implementing EMRs to mitigate potential negative effects [24].

“Improved patient experience” and “improved clinician experience” are 2 quadrants of the Quadruple Aim of health care that warrant significant health service investment amidst the widespread digital transformation of health care. Our findings highlight the need to address pervasive barriers to positive clinician and patient experiences in digital hospitals. To tackle the issues of clinician-patient interaction, inefficient documentation, workload, and burnout identified in this review, health services can invest in feasible and cost-effective solutions such as clinical education and training that are tailored to each clinical discipline as each discipline has unique EMR needs [10]. Prioritizing investment in patient-facing digital solutions such as patient portals can democratize clinical knowledge and empower patients on their unique health care journey [24].

Investment in optimizing EMR infrastructure will build a strong foundation for new clinical applications in descriptive, predictive (ie, artificial intelligence [AI]), and prescriptive (ie, causal AI and decision support) analytics that can benefit clinical workflows and patient outcomes [114]. COVID-19 initiated the rapid adoption of virtual health care [115], and clinical applications of AI are rapidly emerging as the future primary disruptors of health care [116]. Global health services are building machinery to shift from reactive (treat-manage) to proactive (predict-prevent) models of care, with evidence of success for acute clinical problems such as reducing mortality rate and organ failure in the early identification of sepsis [117]. Stakeholder perspectives on implementing clinical AI have been recently consolidated in a qualitative evidence synthesis [118]; however, similarly to our review, patients, carers, and consumers were an underrepresented group compared with clinicians (11.4% vs 70% of data, respectively).

Our approach of using AI (machine learning) via Leximancer to perform the qualitative evidence synthesis is a novel approach compared with recent manual evidence synthesis methods [118-120]. The use of semiautomated content analysis tools such as Leximancer can accelerate progress toward a learning health system. These tools offer an accelerated pipeline for analyzing “big” qualitative data that suffer from traditional yet burdensome manual analytic workflows. Key health care use cases of applying digital tools to routine analytical workflows are patient-reported experience measures [121], unstructured clinical notes in EMRs [122], and social media [123]. Natural language processing and machine learning have been tested to analyze free-text comments from patient experience feedback [124]. Applications of Leximancer can be pushed by investigating how its algorithms can be used in real-world health care to drive continuous cycles of quality improvement with greater speed and efficiency compared with a manual control. Leximancer offers an impartial starting point for content analysis by automating the identification of key concepts and themes that warrant further qualitative refinement by the research team.

The scope of this review was limited to experiences in a digital hospital environment, and thus, the experiences of specific digital systems (eg, telehealth and patient portals) were not considered. The complex, interacting factors that influence experience; the stage of digital hospital implementation; and the differences among settings were not explored in this review and offer important foci for future research. By not including gray literature and articles not published in English, our search strategy may have missed informal evaluations of clinician and patient experience of digital hospitals (eg, within health service annual reports) and geographical variation in digital hospital evaluations. The heterogeneity of digital health environments is reflected in the heterogeneity of studies included in this review, meaning that it is difficult to draw definitive conclusions agnostic to time and place. One limitation of using Leximancer for the qualitative evidence synthesis is that it does not automatically identify emotive concepts; these are identified by researchers when interpreting the results. Although Leximancer reduces the potential for human bias when compared with manual analysis, researcher interpretation of Leximancer results remains a gateway for introducing bias [40]. A qualitative evidence synthesis for patient experience studies was not possible as we only identified one eligible study. The patient experience results should be interpreted with caution because of the relatively limited patient experience data in studies (10/61, 16%) compared with clinician experience data in studies (55/61, 90%).

The clinician experience of digital hospitals appeared positive according to high-level indicators (eg, overall satisfaction and data accessibility); however, the qualitative evidence synthesis revealed substantive tensions between digital hospitals and overall experience, such as weakening clinician-patient interaction, change burden, and inefficient data workflows. There is insufficient evidence to draw a definitive conclusion on the patient experience of digital hospitals, but quantitative indications of satisfaction appear positive or agnostic to digitization. Future research must prioritize investigating the patient experience in digital hospitals and measuring the link between exposure (digital hospital) and outcome (experience) in carefully designed pragmatic trials. Areas of interest include examining the interacting factors that influence experience, the stage of digital hospital implementation, and the differences among settings. Equitable investigation of the patient (including pediatric patients) and clinician digital hospital experience must be prioritized in future research. Worldwide, as digital health becomes inseparable from hospitals and general health care, understanding how to optimize the clinician and patient experience in digital hospital environments will be critical to achieving the Quadruple Aim of (digital) health care.