In the Netherlands, the COVID-19 pandemic led to the first lockdown from March 2020 to June 2020 and the second lockdown from mid-December 2020 to April 2021. Throughout these periods, in-person visits were minimized to reduce the risk of potential virus transmission. As a substitute for in-person visits, the use of video visits significantly increased across many medical specialties.

Video visits were already in practice on a limited scale in a wide range of contexts: for speech evaluation in patients with cleft palate [1], genetic counseling [2], follow-up after facial plastic surgery [3], and postoperative wound assessment [4]. Video visits were found to be satisfactory for patients [5] and enabled empathetic patient-professional relationships remotely [1,6,7]. However, large-scale implementation in daily practice was found to be challenging owing to the multilevel complexity of implementation, where people, organizations, and technology continuously interconnect and develop [8-11]. For example, attitudes and beliefs of individual professionals have been shown to act as both facilitators and barriers in the implementation of eHealth applications [8]. Furthermore, video visits appeared to be particularly successful in follow-up appointments, when a preexisting relationship of trust is established between the patient and clinician [10]. For instance, video visits were more easily adopted in follow-up care after cancer surgery compared with a multidisciplinary context of antenatal diabetes care [10].

Owing to the pandemic-driven, accelerated application of video visits, many clinicians gained experience with this mode of health care delivery. This way of providing care offers several advantages, including saving travel time and costs for both patients and their companions and the efficient use of health care resources, such as outpatient clinic space and support [12-14]. As we move into the post–COVID-19 era, video visits are expected to persist as a routine practice, but large-scale use has seemed to stagnate, presumably owing to a lack of guidance, vision, and attention to patients’ needs, as observed in the United States [15,16]. The use and implementation of video visits are expected to be most successful when tailored to the needs of patients, clinicians, and health care organizations [17,18]. Therefore, it is crucial to understand patients’ perspectives and experiences with video visits and identify specific patient groups that show a greater or lesser degree of appreciation for and suitability to video visits [19]. Large studies with diverse patient populations covering all medical specialties need to be conducted to learn more about patients’ perspectives [16,18]. However, we are concerned that only a limited number of studies have been published that evaluated video visits for patients within large, diverse populations. Consequently, this study contributes significantly to the existing body of knowledge in this area [20-22].

The primary aim of this study was to examine patients’ evaluations of video visits in the context of the COVID-19 pandemic. The secondary aim was to identify patient groups for whom video visits are relatively more suitable, given their positive experiences. With these results, the first step toward patient-tailored choices for type of visit can be made.

A cross-sectional study was conducted from August 2020 to December 2020 at the Radboud University Medical Center (Radboudumc). Zaurus was used as the video visit app, which is compatible with all smartphones and tablets. Patients were invited via email to register and download the app.

Patients scheduled for a video visit were automatically selected based on the registered mode of visit. Links to the questionnaire were sent by an independent research firm (Expoints) on behalf of the Radboudumc. Selected patients received the survey within 8 days after their visit via email to evaluate the visit and collect their sociodemographic details. The survey had to be completed within 2 weeks, and a reminder was sent 1 week after the initial invitation. An incomplete survey could be saved to be completed later (within 2 weeks). No reminder for completion was sent.

All patients aged ≥18 years who received a video visit at an outpatient clinic at Radboudumc were eligible for inclusion. When a patient had multiple video visits in the selected period, the most recent video visit was selected.

Patients were excluded if they had cognitive problems; had difficulties with reading and understanding Dutch owing to a hindering language barrier; were deceased at the time of selection; completed a survey regarding video visits in the 180 days before the start of our study; completed a survey regarding their admission experience or experience with an in-person visit 30 days before the start of our study; or were admitted to the hospital, as priority was given to the patient experience survey regarding admission. In addition, when the video visit was a follow-up visit after giving birth or when the video visit was replaced with another visit modality, the patient was excluded.

All patients participated voluntarily and anonymously in the survey and gave informed consent to use their data in accordance with the General Data Protection Regulation. Ethics approval was requested and waived by the local Medical Research Ethics Committee of Radboudumc (CMO [committee on research involving human subjects] Oost-Nederland; registration number 2021-8415).

A combined survey was used, which consisted of the Patient Experience Monitor (PEM) for adult outpatient experience [23], developed by the Dutch Federation of University Medical Centers, and the patient satisfaction survey for video visits created by Hanna et al [7]. This combined survey was constructed after extensive deliberation by an expert panel. In this process, a literature review of surveys specifically about video visits was performed. The experts found the survey by Hanna et al [7] to be the most suitable for the aim of our study.

The PEM survey was constructed by adapting a validated Picker Institute survey following a comprehensive theory-driven approach of item selection by an expert panel, cognitive interviews with patients, analysis of psychometric properties, and member checking. This survey of 14 items represents eight key domains of person-centered care: (1) fast access to reliable health care advice; (2) effective treatment delivered by trusted professionals; (3) continuity of care and smooth transitions; (4) involvement and support for family and caregivers; (5) clear information, communication, and support for self-care; (6) involvement in making decisions and respect for preferences; (7) emotional support, empathy, and respect; and (8) attention to physical and environmental needs [24]. The Picker Institute surveys are measures for evaluating patients’ experiences in outpatient and inpatient clinical care and have been validated and extensively used in university medical centers in the Netherlands since 2019 [25]. The PEM survey was adapted to the videoconferencing setting by rephrasing the questions. Overall, 2 items were open-ended questions (Multimedia Appendix 1). Quantitative analyses of the PEM survey were based on individual survey items, whereas open-ended questions were analyzed using qualitative methods.

The 13-item survey by Hanna et al [7] is designed according to the principles of survey development for telemedicine to evaluate patients’ experiences with video visits in pain clinics [26]. This survey was translated into Dutch, and 1 item was removed, as it was already covered by the PEM survey (Multimedia Appendix 1, items 16-27). In total, 4 items were rephrased based on the advice of the patient communication experts. Analysis of the survey by Hanna et al [7] is based on an overall sum score, where a higher overall sum score represents greater satisfaction. For correct analysis and to calculate an overall sum score, the 3 negatively phrased questions (items 17, 21, and 23) were reversed (eg, “No, definitely not” was converted to “Yes, definitely”), according to protocol.

Finally, a question was added to assess the visit by assigning a score on a scale ranging from 1 to 10 (with 10 being most positive). The survey was conducted according to CHERRIES (Checklist for Reporting Results of Internet-Based e-Surveys; Multimedia Appendix 2) [27].

From August 2020 to December 2020, a total of 1244 surveys were completed, with a response rate of 28.32% (1244/4392). After excluding 15.27% (190/1244) of the patients who reported that the visit was either a telephone consultation or replaced by telephone after technical difficulties, 84.73% (1054/1244) of the surveys were used in the analysis.

Table 1 shows the patients’ demographics. An equal distribution across age categories was observed. Clinical genetics, neurology, and medical oncology accounted for 65.84% (694/1054) of the total number of evaluated video visits, whereas the distribution across the other medical specialties varied widely. The numbers of first and follow-up visits were equal, with most follow-up visits (480/1054, 45.54%) performed by a known clinician. After a video visit, 36.91% (389/1054) of the patients had to make an appointment for an additional in-person visit or medical examination. A follow-up visit via video was planned in 40.32% (425/1054) of the evaluated video visits. Clinicians from medical oncology and neurology more frequently scheduled an in-person follow-up visit for their patients—18.6% (40/215) and 23.3% (50/215), respectively. Moreover, in 39.7% (69/174) and 14.4% (25/174) of cases, the visits provided by clinical genetics and neurology respectively, were followed by a consecutive visit for additional (diagnostic) testing.

The overall grading for the video visit had a mean of 8.6 (SD 1.3; median 9) of 10. For 5 PEM items, >80% of the patients answered positively, that is, patients waited no longer than 5 minutes, clinicians had read their medical records well, patients received understandable answers, patients trusted the clinician, and patients had enough time to discuss their problems with the clinician. Refer to Table 2 for details about the responses of patients.

For the remaining 6 items, more than 21% stated that the item was either not applicable or answered positively. For instance, 80.25% (829/1033) indicated not receiving any new medication for the question about whether the professional explained the adverse effects of new medication. Analysis of the items in the survey by Hanna et al [7] showed that 91.46% (964/1054) of the patients was satisfied with their video visit, 66.98% (706/1054) found it to be just as good as an in-person visit, and 68.69% (724/1054) would recommend video visits to other patients, as shown in Table 3.

Of 1054 patients, 574 (54.46%) answered all 3 crucial items on the survey by Hanna et al [7] positively, 234 (22.2%) answered 2 of 3 positively, 138 (13.09%) answered only 1 item positively, 72 (6.83%) patients answered “not applicable” or responded negatively, and 36 (3.42%) responses were missing. Patient and visit characteristics of patients who positively answered all 3 crucial Hanna [7] items versus the group who did not are shown in Multimedia Appendix 3.

Results of the binary logistic regression are shown in Table 4. Negative association was found between the positive evaluation of a visit and the surgical and “other” medical specialties (B=−0.64; P<.001). Positive evaluation was also associated with the patient category who described their health as “very well” (B=1.12; P=.01). Sex, age, and educational status had no influence on whether a patient rated the visit positively. In addition, the reason for the visit was not found to have any influence on the positive evaluation of a video visit.

In this study, we comprehensively analyzed evaluations of visits via video to a tertiary clinic made by a large, diverse patient population, including appreciated aspects and aspects for improvement. Most patients (964/1054, 91.46%) evaluated the video visits positively, with significantly more positive evaluations when the visits were provided by a clinician from “other” medical specialties, as compared to surgical and nonsurgical specialties, or when the patient rated their health status as “very well.” The appreciated aspects were mostly at the individual professional level, whereas aspects for improvement were reported at the innovation level itself.

Our finding that high self-rated health of patients is an influencing patient characteristic for suitability of video visits echoes the finding that patients with less complex, more straightforward clinical needs are more suitable candidates for video visits than those with complex, high-risk diseases [10,29]. Similar findings were identified in an oncological study wherein telemedicine was received favorably for low-acuity cancer care [16]. In contrast to findings that video visits appear to be more appropriate when the clinician knows the patient beforehand and when it is a follow-up visit [10], we found that the reason for the visit did not have any influence on whether patients rated the video visit positively. Remarkably, the medical specialty providing the visit was found to be a significant associated factor. Nonsurgical visits were found to be most suitable for telemedicine. One can imagine that these visits are less dependent on physical examination, such as internal medicine or dermatology, for instance, as these specialties can easily review laboratory abnormalities or skin disorders on screen [30]. In addition to specific aspects of the visit that may depend on the medical specialty, other dimensions or elements during the visit could affect the patient experience, such as the communication strategy used by the clinician [31]. Nonetheless, more studies are needed to get a clear overview about whether medical specialty is a truly discriminative characteristic or whether it is more dependent on the attitude of certain clinicians and patient groups.

Qualitative analysis of the open-ended questions revealed both facilitating and hindering factors for broad implementation and upscaling of video visits. Following Grol and Wensing [28], these factors were categorized into innovation, professional, patient, social context, organizational context, and economic and political context levels. Appreciation was mostly centered on the professionals’ skill in adapting communication to the video setup; however, there was scope for improvement among some individuals, as they might benefit from investing additional effort in making visual contact. The way in which the video visits were organized was also highly valued, especially for the provision of technical support to patients as and when needed.

The attitude of the clinician during the video visit was one of the most frequently mentioned aspects for improvement. The bedside manner, which may be better described as the “webside” manner, of a clinician refers to how the clinician behaves, approaches the patient, and communicates during the visit. Clinicians sometimes seem to lack awareness of how their nonverbal behavior looks on screen, as was also shown by a study that analyzed a large data set of >5000 patients [32]. Patients prefer increased expression of nonverbal empathy from clinicians when they show signs of distress. Inadequate nonverbal communication and body language are often reported as barriers for telemedicine adoption [3,33,34]. Despite expert recommendations dating back several years to raise awareness for nonverbal and paraverbal communication, our study also indicates that there is still considerable scope for improvement and training at the clinician level [31,35].

In accordance with several survey studies conducted during the pandemic, our response rate was low. This might be explained by the questionnaire fatigue that was frequently observed during the COVID-19 pandemic, as patients received multiple questionnaires and messages from the outpatient clinic, apart from research [36].

Regarding future perspectives, clinicians should seek guidelines to assess the suitability of a video visit, and the following recommendations might be useful. The Dutch Center of Expertise on Health Disparities recommends checking the patients’ digital skills beforehand, providing digital support, and evaluating whether the information is correctly understood through techniques such as “teach back” at the end of the visit [37]. Video consulting guidelines advise considering several factors while deciding whether video visits may be suitable, such as whether there is an established relationship with the patient, whether it entails nonurgent care, whether there is a need for physical examination, and whether there are factors in favor of the patient staying at home [38].

The findings of this study must be considered in the light of some limitations. First, our study might have been exposed to selection and sampling biases for several reasons. It was an “open” survey, where patients could decide voluntarily whether they would participate in the survey, which might have led to a sample of patients that is not representative of the entire population of the hospital. In addition, owing to the exclusion criteria, not all video visits were evaluated.

Second, the validity of the combined survey was not tested. As the analysis of our data was reported at the item level, calculating the internal consistency using Cronbach α was not applicable. PEM is known to be a validated survey; however, the psychometric properties of the survey by Hanna et al [7] are not known and should be determined. As the PEM items were rephrased to the videoconferencing setting, reliability of this new PEM survey will have to be reassessed.

The educational status of patients attending a university medical center is, in general, often higher than the mean educational status of the general population. In our study population, 44.4% (468/1054) of the patients were highly educated, compared with 30% in the Dutch population in 2018 [39]. As teaching hospitals and referring hospitals might serve different populations, the generalizability of our results might be limited, and further studies including different types of hospitals are recommended.

A total of 139 patients reported a failed video visit and noted that the visit was replaced by telephone. However, the exact number of times this occurred is not known, as not all patients might have reported this failure, which also may have resulted in selection bias.

In this evaluation, we deliberately focused on the patient evaluation of video visits. As it is known that patients and clinicians have different views about quality of information and visits [40-42], the clinicians’ point of view should also be explored for a comprehensive evaluation of the use of video visits. Health care providers, such as clinicians, might experience different barriers and facilitators compared with patients, thus influencing the successful implementation of video visits. Key barriers to successful implementation such as the lack of training and motivation to offer video visits need to be addressed [3,8,43,44].

Video visits were perceived as highly satisfactory by patients during the COVID-19 pandemic, with the best experiences reported by healthy participants and participants who scheduled a visit with a clinician outside the realms of surgical and nonsurgical medical specialties, such as clinical genetics or radiotherapy. Appreciated aspects were mainly at the individual professional level, organizational level, and at the level of the innovation itself. The mentioned aspects for improvement can be changed for the better.

The findings cannot be directly generalized as they were collected in a university medical center with a specific patient population, but they provide additional results for understanding the suitability of video visits in a broad patient population. To be able to truly tailor the use of video visits to patients’ needs, a patient-centered perspective involving both patients and health care professionals is needed.