Online consultation (OC) systems allow patients to contact their health care provider over the internet to ask health-related questions and report symptoms [1]. Their query may then be resolved with a written response, telephone call, video consultation, or in-person visit. Many terms are used to describe this type of technology, including e-consultation, e-visit, and online triage (Multimedia Appendix 1 [2-28])—in this review, we refer to them all as online consultations. We distinguish OCs from “symptom checkers” [29] and other self-service systems that typically do not directly facilitate communication with a human health care provider and from patient portals [30], which may include generic email or secure messaging functionalities.

OCs are considered by policy makers in many countries as a way to address the increasing workload and decreasing workforce capacity in primary care [31-36] while still meeting patient expectations and improving access [37]. However, they have the potential to exacerbate health inequities [38,39] and increase inappropriate antibiotic prescriptions [40]. Furthermore, there are widely recognized challenges in initiating and sustaining the adoption of new technologies in primary care [41].

Although symptom checkers [29,42] and patient portals [30,43,44] have been well studied, only a small number of evidence syntheses directly relevant to OCs have been published: a systematic review of 57 articles on delivering “e-consultation” in primary care largely focused on generic stand-alone applications such as email and video (n=39/57, 68%) [45]; a scoping review of “online triage tools” included 13 papers, 4 of which (31%) were nonempirical (eg, opinion pieces) [46]; and a review of 17 studies of “intelligent online triage tools” focused only on those that used “artificial intelligence” (AI) [47].

Since these syntheses were conducted, OCs have gained wider traction in clinical practice worldwide—they have been indispensable in helping manage patients remotely to minimize the spread of COVID-19 [48,49], and English primary care providers have been mandated to offer OCs for all patients since April 2020 [50]. Moreover, OC system product design has progressed significantly to become more specialized and technologically advanced [51], with several more empirical research studies published on their use [2-11,52-64].

Given this rapid scale-up and increase in the diversity and complexity of OCs, further insight is needed into their impact on health care quality. Previous reviews have not reported the design or implementation details of the OCs they studied [45-47] despite their importance in understanding the causal mechanisms of how they affect care outcomes [65]. The aim of this study was to systematically review and synthesize the empirical quantitative and qualitative literature in a theoretically informed way to address this knowledge gap.

Informed by existing theories, the aim of this study was to synthesize quantitative and qualitative research on (1) the impacts of OCs on primary care quality and (2) how these are influenced by OC system design and implementation.

We consider OCs as complex interventions and, therefore, synthesized both quantitative and qualitative evidence to understand their impacts in specific contexts [66]. We did not perform a meta-analysis because of the heterogeneous and nonrandomized nature of the included studies [67]. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement [68].

The study protocol was registered with PROSPERO (CRD42020191802) [69]. The original title was amended to be less general and more specific to the objectives of the review, and the objectives were amended to focus on care quality.

Papers that met the following criteria were included: empirical studies using quantitative or qualitative methods to examine the real-world use of OCs in primary care in any country, written in English, and published in 2010 or later. We excluded news articles, opinion pieces, literature reviews, non–English-language articles, and literature published before 2010.

We defined OCs as digital interventions that allow patients to contact their primary care provider by inputting “queries” into health care–specific web-based forms [1]. We included symptom checkers and similar self-service systems [54] if at least one of their outcomes directly facilitated contact with a primary care health professional. We included patient portals if they had a secure messaging functionality that used health care–specific forms [54]. We excluded stand-alone generic communication technologies such as email or videoconferencing software.

We searched the Ovid MEDLINE, EMBASE, Web of Science, and Scopus databases during July 2020 (Multimedia Appendix 2 [12,53,56,58-60,63,70-73]). Our search strategy was developed from scoping searches of the literature and drew on search strategies used in related literature reviews [45,46]. We searched the National Technical Information Service, the Health Management Information Consortium, and Zetoc to find relevant gray literature, conference proceedings, and theses. We found further literature through citation mapping and in the reference lists of the included papers, searching during August 2020 and September 2020. SD and TC independently screened titles and abstracts and then full papers for eligibility, resolving differences through discussion at each stage. All literature searches were rerun by SD between November 2021 and February 2022.

We extracted data from the included papers as verbatim text, capturing study characteristics (eg, research design and study setting) and key findings relevant to our research objectives based on the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework [74] (Multimedia Appendix 3). We used the NASSS to capture “a rich, contextualised narrative of technology-supported change efforts and the numerous interacting influences that help explain its successes, failures, and unexpected events” [75]. The methodological quality of the studies was assessed using the Mixed Methods Appraisal Tool (MMAT), which is designed for qualitative, quantitative, and mixed methods studies [76]. We scored each paper using recommended quintile percentages as cutoffs and considered any paper scoring at least 60% as of “good” quality [77]. SD and TC extracted data from 10 papers independently, which confirmed high interrater agreement. Following this, SD extracted data from the remaining papers, which were checked by TC.

The data were imported into NVivo (version 12; QSR International) [78] for synthesis. To integrate both quantitative and qualitative data, during data synthesis, quantitative data were transformed into qualitative themes (“qualitising”) [79].

For objective 1, we considered “impacts of OCs on primary care quality” as consequences of using OCs that could relate to patients, primary care staff, or the wider system [65]. We used thematic synthesis [80], which involved SD and TC coding the text from the data extraction forms independently line by line, developing higher-level themes through regular discussion [80]. Impacts on care quality were synthesized inductively, with emerging themes mapped to the six Institute of Medicine domains of health care quality [81]: safe (avoiding harm to patients from care that is intended to help), effective (providing care based on scientific knowledge to produce better clinical outcomes), patient-centered (care that is respectful and responsive), timely (reducing waits and delays for those who receive and give care), efficient (avoiding waste), and equitable (care that does not vary in quality because of personal characteristics) [81]. Our emergent findings suggested that OCs had both positive and negative impacts and, therefore, theme descriptions were edited to be neutral (eg, safe→safety and efficient→efficiency).

For objective 2, we considered OC “design” as material properties of an OC, such as features and functionality [74], and “implementation” as the way an OC was introduced and used in a particular context [65]. As a design feature, we considered AI as the ability of machines to “mimic human intelligence as characterized by behaviors such as cognitive ability, memory, learning, and decision making” [82]. We synthesized the extracted data using framework analysis [83], which involved SD and TC reading and rereading each data extraction form and then coding them line by line independently—both deductively by using domains from the NASSS framework [74] for high-level themes and inductively by identifying additional subthemes. Through discussion, SD and TC summarized the findings into five high-level themes: condition complexity (health condition and the illness the OC is used for), technology (material properties of the OC and required knowledge for use), adopters (staff, patients, and carers expected to use the OC), organization (extent of work needed for implementation of the OC, capacity, and readiness), and wider system (policy context) [74]. Two NASSS domains—value proposition (value of the OC to the developer, patients, and health care system) and embedding and adaptation over time (learning and adaptation to changing contexts)—had limited applicability to our findings and were not included in the final synthesis. Informed by realistic evaluation [65], we considered our themes as contextual factors and identified patterns of explanations for how each led to the impacts on care quality from objective 1 (ie, “causal mechanisms”). Where appropriate, we considered the levels of OC adoption as a mechanism for how they affected care quality [65]. We used visual mapping to identify commonalities and discordances in causal mechanisms—first within individual papers and then across papers [83]. Where there were discordances, we explored potential explanations where possible (eg, related to the study setting).

The strength and quality of our findings for objectives 1 and 2 were assessed using the Grading of Recommendations Assessment, Development, and Evaluation-Confidence in Evidence from Reviews of Qualitative Research method [84]. This accounts for the methodological limitations of the contributing papers (according to MMAT assessments), relevance to the review question, coherence of the finding, and adequacy of its supporting data [84]. Confidence in each finding was designated as high, moderate, low, or very low. At each stage of the analysis, the findings were discussed and agreed upon with the wider study team. BCB reviewed all coded verbatim excerpts from the papers included in the final synthesis.

We synthesized 63 papers (Figure 1), including 52 (83%) journal papers [53], 7 (11%) evaluation reports [85], 3 (5%) conference papers [12], and 1 (2%) master’s degree thesis [13]. The studies were quantitative (33/63, 52%), qualitative (12/63, 19%), and mixed methods (18/63, 29%) and analyzed data from patients (16/63, 25% qualitative studies and 18/63, 29% quantitative studies), staff (22/63, 35% qualitative studies and 9/63, 14% quantitative studies), and clinical systems (33/63, 52% quantitative studies). All were set in one of 9 high-income countries, with most coming from the United States (21/63, 33%) and the United Kingdom (20/63, 32%; Multimedia Appendix 4 [2-27,52-64,70-73,77,85-104]). In all, 41% (26/63) of the studies were published in 2020 or later, and 17% (11/63) were conducted after the start of the COVID-19 pandemic. Examples of excluded studies are those that focused on stand-alone video consultations [105], involved communication between physicians and not patients [106], and were not based on primary care [107].

In all, 83% (52/63) of the studies reported levels of OC adoption by patients and staff, of which 62% (32/52; 32/63, 51% of all studies) were described as “low” by the study authors [86]. OCs were adopted at a high rate in 63% (33/52; 33/63, 52% of all) of the studies [87], including high rates of adoption by certain patient groups even when overall OC adoption in the study was low [14].

The included papers described 31 OC systems summarized in Table 1 and detailed in Multimedia Appendix 5 [2-27,52-64,70-73,85-104]. In 25% (16/63) of the papers, the OC system was described sufficiently to meet our inclusion criteria but not in enough detail to determine specific design features. Of the 31 OCs described, most (23/31, 74%) offered two-way written communication between patients and staff [88], with a few (4/31, 13%) also offering communication by video [52]. In all, 13% (4/31) did not provide functionalities for staff to reply to patients via the system (ie, one-way communication only [14]). In total, 35% (11/31) required patients to describe their queries solely via multiple-choice questionnaires (MCQs) [89] compared with 13% (4/31) that solely required patients to describe their queries using unstructured free text [56]. In all, 42% (13/31) had a hybrid approach of primarily using MCQs with the option for patients to enter additional free text [90]. No free text OCs offered optional MCQs. In all, 26% (8/31) of the OC systems were integrated with the electronic health record (EHR) [58], and 3% (1/31) allowed patients to schedule telephone or in-person appointments with health care professionals themselves [54].

In total, 54% (13/24) of MCQ-based OC systems exhibited three types of AI: (1) adapting questions they asked patients as they submitted their query in response to previous answers given (10/31, 32%) [91]; (2) prioritizing patient queries based on clinical urgency (4/31, 13%) [54]; and (3) signposting patients to an appropriate care provider based on their query, such as self-care, primary care, or emergency department (3/31, 10%) [8]. These were mostly powered by preprogrammed logic and “algorithms” (10/31, 32%) [54], with the exact AI methodology unclear in the remainder (3/31, 10%) [15].

The methodological quality of most studies (42/63, 67%) was “good” (ie, ≥60% according to the MMAT [77]; Multimedia Appendix 6 [2-27,52-64,70-73,76,85-104]). Common limitations included a lack of detail on whether the OC was administered as intended [92] and small sample sizes [3].

This review focused on how OCs affect primary care quality, as defined by Institute of Medicine domains, for patients, providers, and the wider system, as well as which factors, as specified through the NASSS framework, influence this quality. We synthesized qualitative and quantitative evidence from 63 studies conducted in 9 countries covering 31 OC systems described in detail, with wide-ranging functionalities including AI. In all, 41% (26/63) of the studies were published in 2020 onward, and 17% (11/63) were published after the COVID-19 pandemic. Our main findings were that OCs are safe and have positive impacts on care quality, including increased access to care and decreased patient costs. However, they can have conflicting impacts on provider costs, staff and patient workloads, patient satisfaction, and care equity. We found that the impacts OCs have on care quality are determined by the complexity of the patient queries they are used for, the design of the OC technology itself, the characteristics of staff and patient users, the way OCs are implemented by health care providers, and wider health policies.

Consistent with previous reviews relevant to OCs, we found a limited demographic of patients using OCs, leading to potential inequitable care [45,46]. We also found that the studies often did not sufficiently explore patients’ perspectives of OCs in depth [46]; only 14% (9/63) of the studies used interview-based methods with an average sample size of 24.5 (SD 10.14). This hampered efforts to understand how such inequities arose.

Contrary to previous reviews, we found that OC impacts on care quality are more complex and nuanced than previously reported [45-47]. For example, we identified mixed findings regarding their impact on workload, patient satisfaction, and equitable care. This contrasts with previous reviews, where OCs only increased [47] or had no impact [45] on workload, decreased patient safety [45,47], and increased inequity [45-47].

These new findings for OCs may be partly explained because 76% (48/63) of the included studies had not been covered by these previous reviews. Although there was some overlap of papers (7/57, 12% of papers [45]; 7/13, 54% of papers [46]; and 4/17, 24% of papers [47]), most did not meet our inclusion criteria as they were either nonempirical (4/57, 7% [45]; 4/13, 31% [46]; and 4/17, 24% [47]), published before 2010 (26/57, 46% [45] and 2/17, 12% [47]), not based on real-world primary care (16/57, 28% [45]; 1/13, 8% [46]; and 6/17, 35% [47]), or did not meet our functional definition of an OC (39/57, 68% [45]; 2/13, 15% [46]; and 6/17, 35% [47]; eg, symptom checkers with no link to a health professional [28]).

By focusing on design and implementation, we identified new ways in which OCs affect primary care quality. For example, we found that, by increasing access, OCs can increase staff workload by creating “supply-induced demand” [17,108] and that they can decrease workload by enabling more focused consultations [17]. Furthermore, as previous reviews often did not analyze the design or implementation of OCs [45-47], we identified influential factors that have not been previously described. For example, although some reviews identified increased workload when clinicians received insufficient patient information via an OC system [46], we found that this was particularly associated with MCQ-based OCs [23]. We identified that allowing patients to describe their queries using unstructured free text had the opposite effect [95] while also having a positive impact on patient safety [55]. Using unstructured free text means that patients can more fully describe their query in addition to allowing them to freely express their ideas, concerns, and expectations, as is common in patient-centered primary care consultations [109].

As evidenced by the range of examples in Multimedia Appendix 1, we adopted a fundamental functional definition of OCs rather than relying on the names given to them by the authors of the included studies. When combined with our comprehensive searches across multiple databases and inclusion of gray literature, we identified more empirical studies relevant to OCs than any previous evidence synthesis on the topic [45-47]. Combined with our focus on causal mechanisms, this helped us develop a new and theoretically informed understanding of OCs that has not been previously reported.

As in all systematic reviews, our synthesis is reliant on what the study authors reported. OC features were not always described in sufficient detail to understand how they affected care quality [62]. There was also a lack of patient perspective in the studies, particularly from OC nonusers [4]. We made our literature search strategy as inclusive as possible regarding the different terms used for OCs (Multimedia Appendix 1) but, owing to their wide-ranging nature, it is possible that some papers were missed. We updated our searches between November 2021 and February 2022 to capture more recently published studies but, owing to time constraints, only 1 author (SD) screened these newer papers. This enabled us to capture studies conducted in the context of COVID-19 (11/63, 17% of all included studies).

This is the first theoretically informed synthesis of empirical research on OCs in primary care and uniquely includes studies conducted during the COVID-19 pandemic. It contributes new knowledge that OCs are safe and have positive impacts on care quality, including increased access to primary care and decreased patient costs. However, they are also complex and often produce conflicting impacts on provider costs, staff and patient workloads, patient satisfaction, and care equity. Some of these are unintended and conflict with the promotion of OCs by policy makers as a way to address already increasing workload and decreasing workforce capacity in primary care [31-36]. Unlike previous evidence syntheses on the topic, we have shown that negative impacts on care quality of OCs can be mitigated through appropriate system design (eg, free text formats and two-way written communication), incorporation of advanced technologies (eg, AI), and integration into technical infrastructure (eg, EHRs) and organizational workflows (eg, timely responses). Since the advent of COVID-19, OCs have become indispensable, although further engineering and implementation research is required to realize their full benefits.