The significant health disparities for residents of rural and remote communities compared with metropolitan or urban populations raise questions of equity and access to health services. Multiple reasons have been put forward internationally to explain these health disparities [1-4]. Limited access to health care is seen as a major contributor to rural or remote and metropolitan or urban health disparities, with workforce supply central to this discourse [5-10].

Emergency department (ED) services are an essential component of the health system, often serving as the first or only point of contact for patients requiring medical care. Patients presenting to tertiary center EDs, often in an urban location, can be assured of a well-supported ED with sufficient post-ED care within the same hospital or at another hospital within a short distance [11]. Transfer decisions in rural and remote EDs do not generally have the assurance of timely and appropriate follow-up care. The challenges in rural and remote ED care are 2-fold. First, variations are evident in the capability of local hospitals, arising mainly from the lack of economy of scale to justify investments in a full range of capabilities and inpatient wards for continued treatment and monitoring after completion of ED care. Second, the time needed and distance involved to reach definitive health care compared with suburban or urban settings [12] can delay time-critical treatments such as thrombolytic treatments to resolve a dangerous clot in the blood vessels. This means that, in making decisions around patient dispositions, local hospital capabilities and distance from destination hospitals should be considered together with the patient’s clinical conditions.

Emergency telehealth services provide rural and remote hospitals with timely specialist expertise to increase staff support during critical ED encounters [12], to some extent mitigating the inequities in workforce supply. The key question in the evaluation of the effectiveness of an emergency telehealth service is whether this increased specialist workforce participation in rural and remote ED presentations improves patient outcomes by delivering more timely and effective care.

Only 1 systematic review has examined the use of telehealth in rural and remote EDs, and its focus was on noncritical emergency presentations [11]. The scope of ED services included in the review ranged from telepsychiatry to assist with mental health emergency presentations and teleophthalmology for acute eye concerns requiring ophthalmologist assessment to tele-emergency, half of which involved the use of teleradiology and consultation with other subspecialists [11]. The outcomes of interest were uptake of the telehealth program, change in diagnosis or management plan, patient transfer rate, and patient dispositions (discharge, local admission, and discharge against medical advice) [11]. Of the 15 studies reviewed, 5 reported the influence of telehealth on patient diagnosis or management, with teleconsultations changing the diagnosis or management in 18%-66% of the consultations [13-17]. The review also discussed the dependence of patient dispositions on telehealth program design and observed close linkages between the rate of patient transfer, discharge, local admission, and discharge against medical advice and emergency telehealth use [11]. Most of the studies included in the review reported increases in patient transfer rates [11]. A total of 4 studies aligned telehealth with a reduction in unnecessary patient transfers [15,18-20]. Apparent in the review was the reduction of unnecessary transfers and secondary overtriage (misidentification of noncritical patients as critically ill at initial presentation), which translated into increased local hospital admissions and reduced discharge after teleconsultation [19,20].

Whether reduced unnecessary transfer and increased local admission are favorable outcomes for patients depends on the acuity and health conditions being treated as well as the infrastructure and workforce capabilities of the local hospitals to accommodate the increased demand. For the same reasons, transfer avoidance may not always result in favorable outcomes for patients presenting to rural and remote EDs. Similarly, an increase in local hospital admission may not always lead to favorable patient outcomes if specialist consultations through telehealth alone do not change the capability of local (receiving-end) hospitals to continue caring for patients presenting to the ED in critical conditions. While identifying outcome measures, the systematic review of the use of telehealth in managing emergencies in rural and remote EDs did not consider the relevance of outcome measures across the various clinical contexts such as clinical area and the acuity of presentation. In addition, its focus was on noncritical presentations only [11].

The aim of this systematic review is to examine the outcome measures used in studying the effectiveness of rural and remote emergency telehealth services and analyze the clinical context in which these outcome measures were used and interpreted. The findings from this review provide insight into evaluating the clinical impact of telehealth services in rural and remote EDs and will assist in the design of future studies on the effectiveness and cost-effectiveness of emergency telehealth services in the rural and remote context.

This systematic review followed the effectiveness part of a published protocol on reviewing the literature on the effectiveness and cost-effectiveness of telehealth services in rural and remote EDs [21]. Studies were selected using the Population, Intervention, Comparator, Outcomes of Interest, and Study Design framework (see Table 1 for inclusion and exclusion criteria). Although there was a substantial body of literature on using telehealth to support prehospital emergency medical services, to restrict the scope, this review only included studies taking place in hospital EDs and excluded records reporting on the use of prehospital emergency telehealth.

The search strategy used Medical Subject Headings and equivalent lists of subject descriptors to find articles covering 4 key domains: telehealth or telemedicine, EDs, effectiveness, and rural and remote (Figure 1).

This search strategy was applied to MEDLINE (Ovid), Cochrane Library, Scopus, CINAHL, ProQuest, and EconLit, as well as the Centre for Reviews and Dissemination databases (eg, the National Health Service Economic Evaluation database) for the search period from January 1, 1990, to May 23, 2020. The reference lists of the included studies were hand searched to include other peer-reviewed publications relevant to this review. Finally, a search was conducted on Google using the phrase “effectiveness of rural and remote emergency department telehealth.”

The search of the electronic databases identified 934 records for title screening. An additional record was identified from the Google search. Of the total 935 records, 207 (22.1%) duplicate titles and 528 (56.5%) irrelevant titles were removed. Of the remaining 200 abstracts screened, 165 (82.5%) were excluded because they did not fit 1 or more of the inclusion criteria. A full-text review of the remaining 35 records identified a further 15 (43%) that did not meet 1 of the inclusion criteria. An additional record was identified from hand searching of the reference lists of the included records. Detailed review and data extraction were performed on 21 articles. A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram is presented in Figure 5.

Multimedia Appendix 1 [25-36] summarizes the bibliographic information from individual records, including information related to the telehealth intervention, study details, and the authors’ assessment of the reporting quality. Of the 21 studies, 14 (67%) were of high-quality reporting according to the criteria used in this review.

The studies were categorized according to clinical area and country of implementation (Table S1 of Multimedia Appendix 2 [25-36]). Of the 21 records reviewed, the most frequently assessed service (6/21, 29%) was telestroke [16,25-28,36]. Other clinical area categories included ED services in rural and remote regions [12,19,20,29], trauma [30-32], pediatrics [15,33], mental health [18,34], ophthalmology [14], minor injury [13], cardiovascular [35], and burns [17]. Of the 21 studies, 13 (62%) reported telehealth interventions implemented in the United States [12,15-18,20,27-29,31-34]; the other countries reporting on telehealth interventions were the United Kingdom [13,14], Australia [19,36], Finland [26], Germany [25], Norway [30], and Poland [35].

A further categorization was conducted of the clinical area by operational use of telehealth (Table S2 of Multimedia Appendix 2 [25-36]). Of the 21 studies, 12 (57%) involved a specialist supporting local clinicians [12,14,15,17,19,25-28,33,34,36], 8 (38%) involved a direct consultation by a specialist physician [13,16,18,20,29-32], and 1 (5%) concerned the provision of diagnostic services [35]. Of the 8 direct consultations, 7 (88%) were to a location staffed by a nurse practitioner, whereas 1 (12%) [18] involved a mental health specialist directly consulting the patient in the absence of a local clinician. No study documented an intervention involving remote monitoring in the rural and remote emergency setting.

The studies included in this review identified study cohorts according to whether the patients had telephone, videoconferencing, or face-to-face consultations. In terms of the study design, 14% (3/21) of the studies were intervention studies—of these 3 studies, 2 (67%) were randomized [13,16] and 1 (33%) was nonrandomized [14])—10% (2/21) were qualitative studies [12,30], and 76% (16/21) were cohort studies [15,17-20,25-29,31-36]. There were 3 types of comparisons: (1) comparing before-and-after telehealth interventions at spoke hospitals [12,17-20,29,31] (the comparison of early-phase telemedicine implementation with the mature phase of telemedicine implementation in Kleinrok et al [35] and Bladin et al [36] is also a form of before-and-after comparison); (2) cross-sectional comparisons such as the following: (i) comparison between the videoconferencing consultation (typically referred to as telemedicine) at the spoke hospital and face-to-face consultation at the hub hospital [25,27,28], (ii) comparison within spoke hospitals between cohorts of patients who received telemedicine consultation and patients receiving only face-to-face consultations with a local clinician [13,14,32-34], and (iii) comparison between telemedicine and telephone consultations [15,16,30,33]; and (3) comparison of a telemedicine study cohort with population cohort–based data. A study on patients who had received thrombolysis compared the study group with population thrombolysis data sets [26]. Table S3 of Multimedia Appendix 2 [25-36] lists author names and year of publication against the study design categories and comparison categories. Figure 6 illustrates the different types of comparisons.

There was variability among the hub-and-spoke comparison studies, all of which were related to telestroke. The focus of 10% (2/21) of the studies was on patient disposition after thrombolysis: Wang et al [27] compared patients who received thrombolysis locally with patients who were transferred and then received thrombolysis at the hub, whereas Yaghi et al [28] compared the data of patients who had received thrombolysis locally, were transferred to the hub, and stayed locally. Barlinn et al [25] compared patients who had had a stroke and who had been transferred to the hub after telestroke consultation with patients who had been admitted directly to the hub.

The most significant risk of bias within the studies was related to study design. The risk of bias is high in before-and-after comparison studies because systems change over time, which cannot be controlled [20]. The studies did not report on the details of system change; therefore, this risk of bias could not be addressed in this review. Study designs involving cross-sectional comparisons could not be randomized and experienced selection bias pertaining to severity, with higher severity among the telemedicine groups, that is, tendency to consult emergency medicine specialists when managing patients classified as more severe.

The tendency to recruit greater number of telemedicine cases from larger spoke hospitals diluted the effect of remoteness on the effectiveness of telehealth when the results were interpreted as a whole. This is a significant selection bias in telehealth studies and highlights the gap in the published literature on the impact of telehealth in remote and very remote regions.

Other risks of bias in individual studies related to the sample sizes. Of the 21 studies, 7 (33%) had sample sizes of fewer than 100 patients [14,16-19,26,27]. The studies also reported selection bias in relation to local (receiving-end) clinicians’ preselected cases for telehealth consultations when hospital transfers were considered. This process selected patients whose transfers were more imminent after telemedicine specialist consultations.

A further weakness in many of the included studies was that of attribution. The studies assumed the clinical or service use effectiveness to be associated with the telehealth interventions without considering potential confounders.

Timeliness of the clinical decision and, where appropriate, clinical intervention is critical to the effectiveness of acute ED care. In this review, 2 components that contribute to timeliness were not considered: the impact of prehospital emergency medical services and the distance factor. The decision to exclude studies on prehospital emergency medical services was made to restrict the scope of this review to the effectiveness of emergency telehealth services delivered in hospital settings.

The second limitation is related to the distance between the EDs and the destination of transfer. This, combined with the appropriateness of interhospital transfer decisions, contributes to treatment delays in transit and may affect further patient care (clinical) decisions. We were unable to determine the extent to which telehealth was effective in bridging this gap across clinical areas from the evidence reviewed. Although most studies reported on the distance between spoke and hub hospitals, it was not possible to synthesize the impact of distance on the effectiveness findings using the available information. The small number of spokes included in the studies also made it difficult to make meaningful comparisons in relation to the distance in the studies.

The heterogeneity of the studies in this field rendered the observation of relationships among different levels in the outcomes hierarchy an impossible task. However, based on the findings in this systematic review, future studies or evaluation efforts are well placed to consider an outcomes hierarchy, with surrogate outcomes leading to changes in patients’ functional outcome and mortality in the same study.

This review is the first in rural and remote emergency telehealth to demonstrate the importance of understanding the context in which effectiveness measures are applied in evaluating telehealth. In designing telehealth services in rural and remote EDs, the purpose of telehealth by clinical area and acuity of presentation should be ascertained before setting targets for the telehealth services or program. The context around the benefit of telehealth in supporting more informed clinical decisions and accurate diagnosis, more favorable health service use patterns, and longer-term patient outcomes also has policy implications.

Policy makers should be cognizant of the complexities around, and the limitations of, emergency telehealth in rural and remote settings so as to set reasonable expectations regarding the expected outcomes from this modality of service delivery. The ascertainment of service goals should commence by examining the purpose of telehealth by clinical area, acuity of presentation, receiving-end hospital capability, and the level of definitive care to set appropriate performance targets.

Ascertaining outcome measures to accurately reflect the contribution of telehealth in rural and remote EDs is a complex task. Emergency telehealth studies typically use multiple outcome measures to determine the effectiveness of the services. The analysis in this systematic review has revealed 3 criteria in outcome determination in this context: clinical area, acuity of presentation, and the level of definitive care relative to the hospital of origin. These criteria are useful when defining the favorable direction for each outcome measure of interest.

The findings from this review inform the motivation and expectation of emergency rural and remote telehealth services in the design phase. The evidence from this review indicates that emergency telehealth service adoption has resulted in better service use patterns by improving the diagnosis and making first-line management modifiable by bringing in specialist expertise in emergency medicine. However, the factors that influence clinical decisions but are not modifiable by emergency telehealth, such as receiving-end hospital capability, have not been directly studied in the rural and remote ED context.