The definitions of telehealth and telemedicine often vary across authors. In this manuscript, telehealth is defined as the umbrella term for a set of activities and services performed by health care professionals through telecommunication technologies to “support and promote long-distance clinical health care, patient and professional health-related education, public health and health administration” [8]. Hence, telehealth includes collaboration among health care workers discussing and sharing patients’ information through telecommunication channels, data collection and remote monitoring of patients’ health outcomes through digital wearables, and electronic transmission of prescriptions to pharmacists (e-prescribing). Telemedicine, a subset of telehealth, is strictly defined as the diagnosis and treatment of patients through telecommunication technologies. Nevertheless, the term telehealth will be used throughout the manuscript, except when analyzing or quoting health care workers’ comments, to encompass all health care operations performed through telecommunication technologies.

The comments were collected from Medscape, an online platform that provides health care–related news to health professionals and hosts forums on health-related issues. According to the web analytics company similarweb, the online platform Medscape had more than 21 million visits per month during the summer and fall seasons of the year 2020. Most of the visits were from the United States (53%). The list of countries of the platform visitors included the United Kingdom (4%), Australia (4%), Canada (4%), India (3%), the Philippines (2%), and Brazil (2%). The keywords used to search for the comments were telemedicine, telehealth, televisit, virtual visit, online visit, and video visit. Available data on the characteristics of the commenters were collected as well, including the country of residence, occupation, and medical specialty when the commenter was a physician. Since the comments posted on the platform are publicly available and can be accessed by anyone, the institutional review board of the University of Oklahoma, USA, determined that this investigation did not require ethics committee approval.

The sentiment scores of the collected comments were computed using the R package sentiment, which considers valence shifters (eg, happy vs not happy) and (de-)amplifying words (eg, very worried vs slightly worried) [9]. A mixed-effect model with random intercepts was fitted to assess the association between sentiment scores and the following independent variables: residence of the commenter (US vs non-US), occupation (physician, nurse, health administrator, other) of the commenter, and the year in which the comment was made. The restricted maximum likelihood method was used to estimate the model parameters, and the covariance pattern of the model was set to unstructured.

Examining the prevalence of discussion topics across time frames and commenters’ characteristics can shed light on the opinions or concerns about telehealth that are specific to these time frames and these individual characteristics. Structural topic modeling (STM) was used to identify the topics discussed in the comments. STM allows the analyst to incorporate information about the characteristics (metadata) and content of the documents into the modeling process [10]. The following metadata were used as independent variables to estimate the topic prevalence in the comments: the residence (US vs non-US) and occupation (physician, nurse, health administrator, other) of the commenter, the time frame in which the comment was posted (before vs during the pandemic), and the average sentiment score of the comment. The date of March 11, 2020, the day the World Health Organization declared the COVID-19 outbreak a pandemic, was used as the cut-off date for defining the time period [11]. First, the texts of the comments were processed by removing stop words using the System for the Mechanical Analysis and Retrieval of Text (SMART) stop word list. Then, nouns in plural were converted to their singular forms. No stemming was performed. The appropriate number of topics for the corpus of comments was determined to be between 10 and 25 by examining the values for the held-out likelihood, semantic coherence, residuals, and lower bound [12]. See Figure 1. The appropriate number of topics to extract was determined by selecting the model that yielded both the highest semantic coherence and the highest exclusivity values. The number of topics found to have optimal exclusivity was 13, and thus, 13 topics were selected for modeling. Spectral initialization was used for the topic modeling, which converged after 500 iterations, and 4 single-word comments were dropped during the modeling process. The final model consisted of 13 topics, 910 documents, and a 3252-word dictionary. The correlations among the identified topics were computed with the maximum a posteriori estimates of the topic proportions. A threshold of r=0.15 for the correlation was set to determine whether 2 topics were correlated; that is, 2 topics were considered uncorrelated if their pairwise correlation was less than 0.15. STM was performed using the R package stm [12].

A total of 914 comments made by 705 health care workers were collected from the forums. The dates of the comments spanned from the year 2013 through 2020, and the commenters were from at least 46 countries in 6 continents (Africa, Asia, Australia, Europe, North America, and South America). The majority of commenters were physicians (n=468, 66.4%) and lived in the United States (n=561, 79.6%). The characteristics of the 705 health care workers are presented in Table 1. Of the 914 comments, 583 (63.8%) were posted from the year 2013 up to the day the COVID-19 outbreak was declared a pandemic by the World Health Organization on March 11, 2020, and 331 (36.2%) were posted during the pandemic (ie, mid-March 2020-December 2020). The lengths of the comments ranged from 1 word to 630 words (mean=81.9, median=60.5, SD 80.4).

The mixed-effect model with the comment sentiment score as the dependent variable showed that, in general, comments made by health care workers living in the United States were less positive compared to their counterparts residing outside the United States. There were no statistically significant differences in average sentiment scores among occupations (physician, nurse, health administration professional, other) or across years in which comments were posted (2013-2020). Hence, overall, the sentiment scores of comments posted during the pandemic were comparable to those of comments made before the pandemic. See Table 2, Textbox 1, and Figure 2.

STM identified 13 topics presented in Table 3. The relationships between the prevalence of the identified topics and the characteristics of the comments or commenters are presented in Table 4. There was no correlation among the 13 identified topics, that is, all pairwise correlations among topics were less than 0.15. The 3 most discussed topics were virtual visits (topic 1; 10.2% of comments), which are live encounters between a patient and a health care worker through a telecommunication channel (video, telephone, etc); the potential benefits of telehealth (topic 2; 10%); and criticisms of telehealth (topic 3; 9.4%).

The frequently mentioned benefits (topic 2) included the possibility telehealth offers to reach patients who cannot come to the health care facility for some reasons (living with a disability, residing in a remote area, etc), the decrease in the transportation time and cost for the patient, and the improvement in patients’ access to care and to medical specialists. Some of these benefits were emphasized during the pandemic:

Although the topic on the benefits of telehealth (topic 2) was associated with a positive sentiment score overall, the sentiment score for that topic was less positive during the pandemic compared to its prepandemic value. A deeper examination of the comments revealed that this decrease in the sentiment score may have stemmed from the changes in routine care processes due to pandemic lockdowns and from the technical difficulties encountered by health care workers who had acknowledged the value of telehealth services:

Criticisms of telehealth (topic 3) included the impracticality to perform a physical examination of the patient, which may lead to misdiagnoses, and the overprescription of certain medications by health care providers who used telemedicine (eg, antibiotics). Health care workers who raised these criticisms opined that telehealth devalues “the art and science” of medicine:

Some commenters argued that the resistance to telehealth from some health care professionals may be due the noninclusion of telehealth services into the curricula of training institutions:

Other workers stated that adequate training may help decrease the observed resistance to telehealth services and lead to improved quality of these services:

Other topics raised by commenters included the payment of telehealth services by insurance companies (topic 4; 8.8%) and the legal dispositions necessary for the practice of telehealth (topic 6; 7.6%), especially an interstate medical licence for physicians who practice in the United States. Comments on the lack of proper legal dispositions for telehealth were not specific to health care workers from the United States, however. For instance, a physician from Brazil noted that

Virtual visits (topic 1), criticisms of telehealth (topic 3), telehealth services (topic 5), moving consultations to patients’ homes (topic 8), and the advantages of telehealth for remote areas (topic 13) were more frequently discussed during the COVID-19 pandemic compared to the prepandemic period. In contrast, the benefits of telehealth (topic 2), the practice of medicine (topic 7), physical examination in the era of telehealth (topic 10), and the impact of telehealth on care quality (topic 11) were less frequently discussed during the pandemic compared to before the pandemic. Compared to their counterparts living in other countries, health care workers in the United States discussed insurance payment of telehealth services (topic 4), the legal dispositions for telehealth (topic 6), and physical examination in the era of telehealth (topic 10) more frequently. Non-US health care workers, instead, discussed the practice of medicine in the era of telehealth (topic 7) more frequently than their US counterparts. In general, criticisms of telehealth (topic 2) were less frequently discussed by nurses and health administrators compared to physicians. However, nurses and health administrators discussed issues related to telehealth patient appointments and follow-ups (topic 12) more than physicians.

Although the topic of the impact of telehealth on care quality (topic 11) was associated with a negative sentiment score overall, the score was less negative during the pandemic compared to the prepandemic period, signaling a shift in opinion. Such shift can be seen in comments such as the following posted during the pandemic:

However, the adoption of telehealth services by some health care workers may not outlast the pandemic, as illustrated by this statement posted by a physician:

The objective of this research was to examine the opinions of health care workers about telehealth services and to explore the impact the COVID-19 pandemic may have had on these opinions. The analysis showed that the COVID-19 pandemic did not significantly alter the opinions about telehealth services expressed by health care workers on the discussion forum of the study. Although some topics were more (eg, topic 1: virtual visits) or less (eg, topic 7: practice of medicine) frequently discussed during the pandemic compared to the prepandemic period, the same issues about telehealth services were raised during both time frames. Furthermore, the topic of telehealth benefits (topic 2) was associated with a less positive sentiment during the pandemic compared to before the pandemic. However, the topic of the quality of care of telehealth services (topic 11) had a higher positive sentiment score during the pandemic compared to the prepandemic time frame. This finding may be reflecting the opinion among some health care workers that telehealth can enhance the quality of care only in unusual situations when an in-person encounter between patient and clinician is not feasible, as was the case during the pandemic.

The sentiment scores of comments posted by health care professionals living in the United States were less positive than those of opinions expressed by their counterparts living in other countries. The lack of additional information about the characteristics of the commenters (eg, sociodemographics, computer knowledge) did not allow the assessment of whether this association between location of residence and sentiment scores was due to differences in sociodemographics or in the knowledge of information technology (IT) between commenters from the United States and their counterparts from other countries. Indeed, previous studies have reported that the attitude toward technology may differ by age, occupational seniority, and level of training in IT [13,14]. It is worth emphasizing that the general trends in sentiment scores do not necessary reflect sentiments about telehealth per se, as demonstrated by the multiple topics and the associated sentiment scores uncovered by the analysis. Rather, these general trends show the aggregate sentiment of the concerns or expectations elicited by discussions of telehealth.

Although the pandemic may have offered the opportunity to some health care workers to change their opinions about telehealth services, criticisms of these services were still relatively prevalent among the commenters during the pandemic. Reports on the drawbacks of telehealth, such as the overprescription of antibiotics in direct-to-consumer (DTC) telehealth visits, were frequently mentioned by health care workers who had an unfavorable opinion of telehealth services [15]. DTC telehealth providers offer virtual consultations with health care workers to customers or patients through applications installed on the IT devices of these patients. Patients enter their symptoms or motives of consultation in the application and are then assigned to a health care worker in the network of the DTC telehealth company. The interaction between the patient and the health professional is entirely virtual, and the latter may prescribe a treatment at the end of the encounter. DTC telehealth has often been associated with reduced care quality [16]. It should be mentioned that other types of telehealth services that involve some degree of clinical or paraclinical examination have yielded good quality care [17-19]. Another disadvantage of telehealth that was frequently mentioned before and during the pandemic was the inability to perform a physical examination. Conducting physical examinations has been reported to be perceived by health care workers as part of their identity [20]. Hence, telehealth platforms that do not offer the possibility to perform a physical examination are perceived to threaten that identity. Statements such as “telemedicine makes a mockery of the art of medicine” can be interpreted through the lens of the perceived threat to this occupational identity. It is worth noting that some solutions exist to address the issue of physical exam during telehealth sessions. These solutions include facilitated virtual visits, which require the patient to be examined in a designated facility (originating site), usually by a health care professional (facilitator), during the virtual visit. Information about the physical exam is then transmitted by the facilitator to the patient health provider participating in this telehealth session. Another solution for addressing the issue of physical examination is the use of diagnostic tools (eg, digital stethoscope) that can transmit data from the physical examination remotely [21].

Researchers have shown that the technology acceptance model [22], which posits that the individuals’ perceptions of the technology’s usefulness and ease of use influence their acceptance of that technology, was an adequate framework for predicting health care professionals’ intention to use telehealth services [23,24]. Hence, promoting the usefulness of telehealth services and implementing systems that are perceived as easy to use can boost the adoption of telehealth by health professionals. In this study, topics related to the ease of use of telehealth included not only its limitation for conducting a physical examination (topic 10) but also the lack of appropriate legal and service reimbursement dispositions for its adoption (topics 4 and 6). Hence, enhancing telehealth adoption requires addressing these barriers to its usage. Topics related to the perceived usefulness of telehealth included discussions of its benefits (topic 2) and the advantages it offers for the care of patients living in remote or rural areas (topic 13). However, the promotion of the usefulness of telehealth will need to address the perceived drawbacks of its adoption, namely its reported or perceived negative impact on the quality of care (topic 11) and the practice of medicine (topic 3).

Serrano and Karahanna [25] proposed that a successful session of a telehealth visit requires 3 ingredients: the ability of the patient to relay the relevant information through the technology, the competence of the clinician in eliciting from the patient the appropriate information needed to perform the clinical service, and the capacity of the technology to transmit and present the relevant information to both the clinician and the patient. Health professionals’ frustrations from telehealth may come from the patients’ lack of knowledge on how to use the technology to convey information relevant to their condition. A physician wrote,

Hence, educating patients on how to use a telehealth platform should be among the measures implemented to improve the quality of telehealth services. Health care professionals providing telehealth services should be trained as well, not only on how to navigate the technological features of the platform, but also on how to foster a virtual environment that promotes a good patient-clinician relationship that can help obtain relevant clinical information from the patient. As demonstrated by Serrano and Karahanna [25], a clinician task-specific capability (eg, competency in patient history enquiry) can compensate for the limitations of technology in performing certain tasks. In addition, the technology should fit the clinical task to be performed. Many of the criticisms against telehealth from clinicians included the difficulty or impossibility to obtain certain information through the platform. Under the task-technology-fit framework, the platform used for the telehealth should include features that enhance image or sound quality so that the virtual patient-clinician encounter can be as close as possible to an in-person interaction [25]. Additional technological requirements include a secure and adequate internet connection and easy-to-use software [21]. There are platforms that allow performing a certain number of physical examinations (called provider access software), but their capabilities are still limited [21]. Finally, a technological support team should be available when the system malfunctions.

Factors that may improve health care workers’ perceptions about telehealth include involving them in the design and implementation of the organization’s telehealth platforms [26], having opinion leaders express their support for these services and making resources and technical support available [27], and training adequately workers in utilizing these services [28].

The absence of adequate legal dispositions and the differences in telehealth service payment policies by health insurance companies, as noted by some commenters, can hamper the adoption of telehealth [29]. Adler-Milstein et al [30] reported that in the United States, the promotion of private payer reimbursement policies for telehealth services enhances telehealth adoption, while the requirement of an out-of-state licensure for telehealth decreases its adoption. Hence, wider structural changes beyond individual health care organizations are needed as well to foster the adoption of telehealth services.

The study had some limitations. First, the sample of health care workers who posted their comments on telehealth are not necessarily representative of the population of health care professionals working in the United States or in other countries. In addition, the relatively small number of comments collected may not have been exhaustive enough to identify other prevalent viewpoints about telehealth. Finally, there is no available method for determining the exact number of topics to be extracted from a corpus. Methods available are all approximate and use different criteria for selecting the number of topics to extract. Thus, it is possible that methods different from the ones used in this paper (semantic coherence, held-out likelihood, and exclusivity) may have yielded a different value for the number of topics to be extracted. Despite these limitations, the analysis of the collected comments provided a valuable insight into the opinions of health care workers about telehealth services over a period of 8 years (2013-2020).

The COVID-19 pandemic and the lockdowns mandated by public officials to control the spread of infection have fostered an interest in expanding telehealth services among health care organizations. However, hurdles to the widespread adoption of these services still remain, including some health care workers’ resistance to telehealth, insufficient or inadequate legal dispositions for providing these services, and a lack of coverage of these services by insurance companies. Promoting changes in the unfavorable attitude toward telehealth among health care professionals may require the promotion of evidence-based literature that demonstrates high satisfaction of both patients and clinicians about telehealth. Furthermore, including telehealth service training into the curricula of health care professional training institutions can help prepare these workers perform these services.