This study used a convergent parallel mixed methods evaluation. In this study design, quantitative and qualitative methods have similar relevance, the data are collected in the same phase of the research project, the analysis of the data is independent, and the findings are combined and interpreted together [10]. We chose this design to better understand participants’ experiences. Quantitative data consisted of in-person and telemedicine visits, patient demographics, satisfaction, and challenges identified by clinicians. Data from patients receiving telemedicine care were compared to that of patients receiving in-person care between March 1 and April 30, 2019 (retrospective control group), and March 1 and April 30, 2020 (concurrent control group). Qualitative data included physicians’ responses to open-ended questions asking about challenges related to using telemedicine and mechanisms to address those challenges when delivering virtual health services. This study was approved by the Institutional Review Board of the Pontificia Universidad Católica de Chile.

Chile’s health system combines public and private health providers. Public health services are paid for by public health insurance. Private health providers are paid per service delivered, and health insurance pays some or all of the fees charged by the health care providers.

This study was conducted at the UC Christus Health Network, a large private academic health network in Santiago, Chile. This health network has two hospitals with nearly 500 beds in total and 6 primary and secondary care clinics, with more than 800,000 patient visits every year. UC Christus has been nationally recognized for its innovation and patient service with the “Best Place to Innovate,” “Praxis Xperience Index,” and “ProCalidad” Awards.

UC Christus Health started implementing telemedicine on March 17, 2020, about two weeks after the first patient with COVID-19 was diagnosed in Chile. Initially, it started with generalist specialties, including internal medicine and family medicine, which offered services and guidance for patients with respiratory symptoms. Telemedicine was subsequently implemented as a strategy to provide care across the entire health care system. Currently, more than 720 providers from 61 clinical specialties are providing patient care through telemedicine.

To access these encounters, patients schedule appointments online or over the phone. When scheduling the appointment, patients must accept the terms of telemedicine, including that providers cannot provide certain services virtually, such as ordering restricted prescriptions or providing official health certificates. Once the appointment is scheduled, patients receive a confirmation email and are invited to pay for their visit. The cost of this service is about US $50, representing about two-thirds of the cost for in-person visits. Once the visit is paid for, patients and providers receive a Zoom link (Zoom Video Communications Inc) for their telemedicine visit, and they connect at the scheduled time. An administrative team supports patient-provider connections. Providers have access to the patient's electronic medical record and use a separate platform to order prescriptions, laboratory exams, images, and procedures, which are usually written manually during in-person visits. These electronic orders are emailed automatically to patients from the medication and laboratory platform.

In 2019 and 2020 before COVID-19 arrived in Chile, there were an average of 3039 and 3163 daily outpatient visits in the UC Christus Health Network, respectively. In-person visits were reduced to an average of 384.7 daily visits 3 weeks after the first patient with COVID-19 was confirmed in Chile, representing an 87.9% reduction in outpatient visits (Figure 1, P<.001). Since telemedicine services began, the number of daily visits has increased, with the UC Christus Health Network delivering up to 509 virtual visits each day (P<.001).

Figure 2 displays telemedicine and in-person activity across medical specialties during 2020 compared to 2019. All specialties except psychiatry decreased the number of in-person visits. Pediatrics subspecialties, along with ophthalmology, otorhinolaryngology, and dermatology had the greatest reduction of in-person visits, experiencing between 61.6% to 70.0% of the demand of the previous year (P<.001).

Physicians from all specialty groups were able to implement virtual visits. Even though psychiatrists have been able to provide up to 24.7% of their 2019 volume of clinical encounters through virtual care, most medical groups have only delivered between 2.8% and 10.8% of visits compared to their 2019 baseline (P<.001).

Table 1 compares the demographic characteristics of the telemedicine, concurrent control, and retrospective control groups. Patients accessing telemedicine were more likely to be female and have private insurance, and less likely to be children or older adults, or residents of the Santiago Metropolitan Region compared to both in-person control groups (P<.01 for all comparisons).

During March and April of 2019 and 2020, 1848 and 1187 patients responded to the satisfaction questionnaire after the in-person visits, respectively. In addition, 3962 patients responded to the telemedicine satisfaction assessment. Satisfaction was very high with both telemedicine and in-person services (Figure 3). Patients receiving telemedicine services reported similar satisfaction with clinician’s services compared to both control groups. Patients using telemedicine reported similar access to providers compared to in-person visits in 2020 (P=.08), but greater access compared to in-person visits in 2019 (8.7% increase, P<.001). Patients using telemedicine care reported less satisfaction with the payment process (5.3% reduction, P<.001) and infrastructure (web portal, 3.4% reduction, P<.001) compared to patients receiving in-person visits concurrently. Ratings in these domains were not statistically different compared to patients receiving in-person visits in 2019.

A total of 263 clinicians responded to the satisfaction assessment (36.5%). They were mostly female (n=155, 58.9%), with an average age of 44 years (SD 10.9), and an average of 16.8 years (SD 11.4) of clinical experience. Most providers reported limited experience with telemedicine, with 147 clinicians (61%) reporting 10 or fewer virtual patients encounters at the time they responded to the questionnaire. There were no differences in gender (P=.37), age (P=.10), or specialty category (P=.32) among survey respondents and nonrespondents (n=721 providers). Overall, 244 providers were satisfied or very satisfied with telemedicine (92.8%), and most would recommend this service to friends or family members (94.2%).

When providing telemedicine care, most physicians felt their clinical skills challenged somewhat or a lot (61.8%). Female providers felt more challenged than male providers (70.7% versus 50.9%, P=.002). Surgeons, obstetricians, and gynecologists felt that their clinical skills were challenged the least, compared to providers from other medical specialties (P=.02). These associations persisted after adjusting for covariates in a logistic regression model (Figure 4). There were no statistically significant differences in feeling that clinical skills were challenged according to the provider’s age, years of clinical experience, or the number of patients seen virtually.

Table 1 in Multimedia Appendix 1 summarizes the categories and subcategories of challenges experienced with telemedicine delivery, the diagnostic process, and the patient-provider relationship. Most challenges were related to the delivery mode, specifically accessing the platforms used, dealing with patient scheduling, and accessing the resources required for telemedicine. There were no challenges specifically related to patient treatment. Several strategies were implemented by physicians to address these barriers (Table 2 in Multimedia Appendix 1). Strategies most commonly used were directly contacting the patient, and requesting support from the help center, peers, or patients.

Figure 5 represents the rates of physicians experiencing any challenge with the delivery modality, diagnostic process, and patient-provider relationship according to their specialty category. Providers from all specialties experienced challenges with the delivery system, especially providers from internal medicine subspecialties and psychiatry (Figure 5A, P=.046). There were differences by specialty when reporting challenges related to a patient’s diagnostic process. Most physicians who experienced challenges were from primary care and pediatrics subspecialties (Figure 5B, P<.001). Challenges in the patient-provider relationship also differed by specialty group. About 60% of psychiatrists reported experiencing challenges in this domain compared to 16.4% of surgeons (Figure 5C, P=.02).

The COVID-19 pandemic has radically transformed clinical care and our organization was forced to quickly deploy telemedicine services without the standard change management and training strategies that are usually required for successful digital transformations in health care [14]. Despite this, we have seen a quick uptake and the practice has been widely accepted in a short period of time [15]. According to Kotter’s change management process, the first step toward successful change is the sense of urgency to change [16]; it is undeniable that the COVID-19 pandemic has generated an enormous sense of urgency to adapt to this reality. In a few weeks, we were able to include all clinical specialties in the telemedicine project. Moreover, the Chilean National Health Fund (FONASA), which had delayed including clinical specialties in funding schemes for years, established insurance coverage for telemedical services three weeks after the first COVID-19 case in Chile [17]. The COVID-19 pandemic created the perfect environment for providers and health care organizations to bring down long-lasting barriers to the implementation of telemedicine and other digital innovations [18].

The fear of becoming infected with COVID-19 in health care settings drove many patients away from in-person care [19]. Specialties that focus on children or require medical equipment (eg, otorhinolaryngology or ophthalmology) experienced the highest reduction of in-person visits. This is consistent with recent studies that identified a lack of access to laryngoscopes as a downside of telemedicine visits [20]. Even though telemedicine has provided physicians with an option to deliver outpatient services, the recovery of outpatient clinical activity levels toward 2019 levels differed between specialty groups. Psychiatrists and internal medicine subspecialists have been able to implement the highest number of virtual visits. This different level of adoption could be attributed to patient or clinician characteristics and could be further explored in future studies. In addition, differences were observed in the demographics of patients accessing telemedicine services compared to in-person visits. To continue the growth of this modality of care among all medical groups, and reduce disparities in health care access, further innovations should be developed to provide access to the patient and provider groups that have unequally benefited from this clinical transformation. For example, physicians could use patient-controlled devices to supply equipment normally available in offices [21], and marketing can be focused on groups with less access to care. It is noteworthy that telemedicine increased access to specialty care that previously could have been limited by geographic barriers [22].

Overall, both clinicians and patients were highly satisfied with the implemented telemedicine services. This is partially comparable to the findings from a previous systematic review that synthesized 17 studies assessing patient and physician perceptions of telemedicine versus in-person consultations in different medical specialties [23]. The systematic review found that most studies reported no significant differences in patient-related outcomes such as convenience, format of the consultation, or rapport. In addition, the same systematic review found that overall, physicians showed reduced satisfaction with telemedicine visits regarding communication (patient-provider relationship), and physical examination or diagnostic assessment. These findings are compatible with the results of our study, as some of the concerns identified by providers related to the impact of telemedicine on diagnostic assessment and the patient-provider relationship, as well as complications associated with the implemented delivery modality. Even though the system was far from perfect (eg, using three different platforms during a virtual encounter), patients and providers valued telemedicine as an innovation that mitigated the risk of infection associated with in-person visits. Reducing the risk of contagion has been identified as a likely facilitator to adopting telemedicine [24].

In parallel with a high overall satisfaction with telemedicine, we observed that more than 60% of all clinicians felt that their clinical skills were challenged during virtual visits. This is similar to a previous study that reported an increase in overall workload, mental effort, and psychological stress during virtual visits [25]. We also observed differences across specialties in the perception of telemedicine consultations and how telemedicine can challenge their clinical skills; those in technical specialties found telemedicine less challenging than those in relationship-focused specialties [26,27]. This difference in the physicians’ orientation toward patient care might also reflect differences in adjusting to a new service delivery process.

When analyzing the types of challenges faced by clinicians and the strategies used to address them, it is noticeable that most of the challenges faced were due to the delivery mode, the platform used, and patient scheduling. This finding was identified by embedding a qualitative assessment to the quantitative survey and is consistent with a Cochrane systematic review that found that technical difficulties experienced by clinicians during telemedicine services produced high dropout rates [28]. As telemedicine was quickly deployed as a system-wide innovation in response to COVID-19, there was no time to produce a user-friendly platform, but this will be fundamental to maintaining clinician engagement with virtual services as the pandemic evolves as well as once the pandemic is over. In addition, for telemedicine to overcome all identified challenges, further training is needed among providers. Although medical schools have embraced online training [29], medical students should also be trained in telemedicine [30]. The presented system-wide implementation required that all physicians, regardless of their specialty, were knowledgeable about virtual care. As providers become more experienced conducting telemedicine encounters, it is likely that these barriers will be reduced over time and their experience should be considered as valuable input for future curriculum design.

In the patient-clinician relationship domain there were striking differences between specialty groups. This is especially relevant since psychiatrists experienced the greatest relative increase in telemedicine consultations. We did not explicitly seek to study the causes of this difference, but it is probably a consequence of the differing types of clinician-patient relationships between specialties [26,27]. This is relevant since mental health is one of the areas that has seen the greatest increase in uptake in the past years [31] and it has been a major concern during the COVID-19 pandemic [32]. Mental health telemedicine services are, on average, equally effective when compared to face-to-face consultations [4,28].

Although the implementation of virtual care in response to COVID-19 has been reported in different settings [24,33-38], this is the first report of a system-wide accelerated implementation of telemedicine services using quantitative and qualitative methods. This large-scale implementation allowed us to identify and compare visits and challenges experienced by different specialty groups. However, as any other evaluation, this paper has limitations that are important to acknowledge. First, although there were no differences in the demographics and medical characteristics of providers who did and did not respond to the satisfaction survey, selection bias is always present in voluntary surveys [39]. This limitation also affects patient satisfaction ratings. In this group, because the available data consisted of anonymous ratings, we could not assess potential responder bias. Second, because of the rapid deployment of telemedicine services, we developed a patient satisfaction questionnaire to assess patient satisfaction with telemedicine similar to the NPS but could have used validated measures [40] and integrated qualitative components, as we did in the developed questionnaire for providers. Finally, COVID-19 not only affected health care provider priorities but also changed patient priorities. For example, during the pandemic, patients may be less likely to seek care for chronic conditions, and this could affect patient satisfaction. Future system-wide evaluations of telemedicine should be conducted once the current pandemic is controlled. Despite these limitations, this report highlights the challenges related to implementing telemedicine experienced by multiple groups of physicians and the mechanisms they used to address these challenges during the COVID-19 pandemic.

Telemedicine produced high satisfaction among patients and providers. Although this modality of clinical care was rapidly deployed in response to the COVID-19 pandemic, there was high heterogeneity in its implementation across medical specialties. These differences need to be considered in future implementations of telemedicine when the current medical context is addressed.