The characterization of telemedicine content in relation to COVID-19 was evaluated across 4 components of tweets: the number of favorites and retweets, language used within the tweet, sentiment of the tweet, and authorship. Tweets in English language posted between January 1, 2020, and April 24, 2021, containing at least one of a series of telemedicine-related terms and at least one of a series of COVID-19–related terms were curated from the Twitter full-archive search available via the Academic Research product track [14]. Search terms were derived from the literature and the Medical Subject Headings thesaurus established by the National Library of Medicine (NLM) of the National Institutes of Health (NIH) (Multimedia Appendix 1) [11,15]. A random sample of tweets containing only COVID-19–related terms was extracted for the same period for comparison with the distribution of sampled tweets matched with the daily distribution of telemedicine-related COVID-19 tweets. No geographic restrictions regarding the location of the tweet were implemented, and all tweets beginning with “RT @” (indicating retweet status) were removed prior to analysis.

Standard natural language processing preprocessing procedures were performed prior to textual analysis of the tweets. First, links, mentions, hashtags, and HTML escape characters were removed from all tweets. Next, tweet-level sentiment was calculated using the Valence Aware Dictionary and sEntiment Reasoner (VADER), which was designed specifically for use on a social media corpus and has been validated in other Twitter-based studies, including one on telemedicine [16,17]. In addition to the compound sentiment score, a categorical sentiment was assigned to each tweet based on prespecified cutoffs within the literature (ie, positive if the compound sentiment was greater than 0.05, negative if the compound sentiment was lesser than –0.05, and neutral if the compound sentiment was between these values) [16]. After determining the sentiment, other preprocessing included tokenizing and lemmatizing the text and removing traditional English and Spanish stop words. In addition, words with an inverse document frequency in the 0.05th percentile were removed given their high frequency across all tweets (eg, “covid19”). Other words removed included non-English words (as determined by the GradyAugmented dataset [18]), as well as words with alternative connotations that may have skewed any analysis (eg, “trump” could refer to the verb or the 45th President of the United States; “patient” could refer to the adjective or a person receiving medical treatment). This processing was performed for both single words as well as bigrams (ie, two-word phrases) and trigrams (ie, three-word phrases).

In addition to evaluating all telemedicine-related COVID-19 tweets (“telemedicine-COVID”) relative to a general COVID-19 sample (“general-COVID”), the authors of tweets with the 100 most and the 100 fewest retweets were extracted and manually labeled with a domain (eg, “news,” “political,” “health and medicine”) and account type (ie, “organization” or “individual”). Domains were created by reviewing the author description information and, for verified users, confirming their identity through an independent Google search. Individual authors who represented a nonverified person were automatically labeled as “private citizen,” whereas all organizations were labeled with a domain regardless of the verification status. This process was completed by one member of the research team for both the telemedicine-COVID and general-COVID data sets to compare outcomes within these tweet subpopulations. A second member of the research team independently labeled 10% of the authors as a “validity check” [19]. This subsequentially resulted in Cohen κ=0.66 (percent agreement: 77.3%) for domain type and κ= 0.75 (percent agreement: 86.4%) for the account type.

A comparative analysis of retweets and favorites by month and by data set (ie, telemedicine-COVID vs general-COVID) was conducted using zero-inflated Poisson regression to accommodate for the preponderance of tweets receiving no favorites or retweets and the count nature of the outcome variable. Sentiment analysis was also performed to assess the distribution of positive, negative, and neutral tweets both overall and by month and compared using chi-square tests. This analysis was repeated to compare sentiment in the telemedicine-COVID data set with that in the general-COVID data set. Chi-square tests were also used to compare the distribution of sentiment between the top 100 most and least retweeted posts both within each data set and between them. All analyses were conducted in Python (version 3.7.4) within the Jupyter Notebook graphical user interface (GUI) (version 7.19.0) and R (version 3.6.3) within the RStudio GUI (version 1.3.959). Code is available on GitHub [20]; tweet IDs are available upon request. Given the public nature of social media data, institutional review board approval was not required as specified in Regulation 45 CFR 46 as authored by the US Department of Health and Human Services Office for Human Research Protections [21].

To the best of our knowledge, this study is the first to comprehensively evaluate tweets pertaining to telemedicine and COVID-19 posted between January 2020 and April 2021 compared to a general COVID-19 data set. Analysis of retweets and favorites suggested monthly variations in the “attention” received by telemedicine-COVID tweets, and these tweets had a significantly lower average number of retweets than the general-COVID data set. Telemedicine-COVID tweets were predominantly positive both overall and by month, especially compared to the general-COVID data set. There were also variations in the frequency and sentiment of tweets made by entities in various domains, including private citizens, politicians, and journalists, and a higher proportion of the most retweeted tweets in the telemedicine-COVID data set were positive than those in the general-COVID data set. The findings presented here demonstrate how social media can be leveraged to perform surveillance of shifting opinions surrounding critical health technologies, including telemedicine.

The number of tweets that mentioned telemedicine and COVID-19 drastically increased between February and March 2020, coinciding with the declaration of the COVID-19 pandemic by the World Health Organization on March 11, 2020 [22]. As stay-at-home orders continued throughout April and May 2020, telemedicine became more prominent within the COVID-19 dialogue on Twitter, and health systems began to adapt to the use of this technology. Discussions surrounding telemedicine decreased through June 2020 and remained relatively constant thereafter, perhaps reflecting the achievement of “steady state.” This leveling off coincides with previous studies that have found that the weekly rate of telemedicine consults in a Medicare population peaked in April before declining in June 2020 [23].

As the volume of telemedicine-COVID tweets evolved, so did the sentiment of these tweets. Over half of all months between March 2020 and April 2021 had a majority of telemedicine-COVID tweets labeled as positive, and all months during this period had a significantly higher proportion of positive tweets compared to the general-COVID data set. Although this finding aligns with a study prior to the pandemic that found a higher number of positive telemedicine tweets than negative and neutral tweets, it contrasts with another study that found that 59% of individuals were either unsure of telemedicine or considered it subpar to traditional care delivery mechanisms [17,24]. The positive results found in this study may be reflective of increasing acceptance toward telemedicine—while patients may have viewed telemedicine as just an alternative to in-person care before the pandemic, telemedicine may now be viewed as an alternative to no care whatsoever. This finding aligns with recent patient surveys on their opinion of telemedicine, which found that 79% of patients were “satisfied” with their experience with telemedicine and 78% felt that they had a health problem that could be addressed virtually [25]. Furthermore, increased positivity in the middle stages of the pandemic (particularly in August and September 2020) reflects the evolving understanding of the Twitter population that reduced contact during a telemedicine visit can provide a safer experience during a highly contagious disease outbreak. The increase in positivity in this period in particular also corresponds to the signing of the Executive Order 13941 on August 3, 2020, which aimed to “improve rural health and telehealth access” to Medicare beneficiaries during the postpandemic period [26,27].

The most followed accounts within both the telemedicine-COVID and general-COVID data sets predominantly consisted of journalists and politicians. These groups present stark contrasts in content veracity—journalists likely provide neutral content with minimal bias (depending on the agency), whereas politicians likely share more polarized content that reflect their own views. Prior work evaluating tweets on telemedicine during the COVID-19 pandemic (but not necessarily related to it) found that private citizens frequently retweeted content from both sources, although politicians were retweeted more frequently [11]. Thus, there is a chance that a larger number of Twitter users’ opinions on telemedicine and COVID-19 may be limited to “echo chambers” that reinforce their own opinion and, in a worst-case scenario, spread misinformation with deleterious consequences. However, the fact that a majority of tweets within the telemedicine-COVID data set were positive suggests that this may not necessarily be the case for tweets pertaining to telemedicine or other novel technologies. Conjecturally, this may reflect that positive news and experiences on telemedicine were amplified during the COVID-19 pandemic, which could lead to more widespread adoption and uptake of this technology.

Telemedicine is the pinnacle “21st century approach” to deliver convenient and less expensive care, and over 50 large US health systems have integrated it into their standard operating procedures [5]. Although the surge in telemedicine visits during the early phases of the pandemic have waned to some extent, evidence shows a high (but geographically variable) degree of persistence of telemedicine, from 8% to almost 48% [23]. This can likely be attributed to relaxations in policies that presented major challenges to telemedicine uptake, including reimbursement parity, interstate licensing, prescribing practices, the use of Health Insurance Portability and Accountability Act (HIPAA)-compliant technologies, and the definition of an “originating site” (ie, where the patient is located), to name a few [28,29]. The overwhelming and consistent positive nature of conversations surrounding telemedicine on Twitter as presented here, coupled with a surge in uptake throughout the pandemic, demonstrate that diverse, heterogeneous populations view telemedicine favorably, including patients, providers, and politicians. These findings provide clear evidence for policymakers that replacing restrictive policies with long-term, systematic favorable toward telemedicine would be met with support from numerous, diverse communities.

Despite the promise of this study, it is not without its limitations. Although the list of search terms was comprehensive and based on prior literature, it may have missed tweets that used other terminology to describe telemedicine-related services. Similarly, it is possible that some of the included tweets may not have directly been related to telemedicine. The analysis also does not include telemedicine-related tweets that were made during the study period that did not contain a direct mention of COVID-19. Although this ensured that the data specifically focused on the impacts of telemedicine as they related to COVID-19, future work could analyze how telemedicine tweets that directly mentioned COVID-19 varied from those that did not during the same time. Furthermore, future work could evaluate whether the trends observed in the varied from tweets made during the same period in the prior year. In addition, the present work does not include geospatial data, which has previously been shown to be an influential component of tweet sentiment [30,31]. Thus, future work could evaluate geospatial variations in telemedicine sentiment, including whether it is associated with uptake of the technology in local medical facilities. In addition, only the top 100 most and least retweeted posts were included in the author-level analysis, and future work could expand on this to label more accounts, improve the classification of labels (eg, labeling fewer accounts as “private citizens”), or analyze only a subset of these accounts (eg, politicians or “influencers”). Lastly, the Twitter population of predominantly 18-24 years old, well-educated individuals is not representative of a US or broader population [32]. Thus, future work is needed to characterize the change in sentiment within other populations, especially those that may not be technologically literate and may therefore encounter barriers when attempting to utilize telemedicine.

Opinions on telemedicine and COVID-19 on Twitter have increased in popularity and were largely positive throughout 2020 and the beginning of 2021. These telemedicine-COVID tweets were generally more positive than general-COVID tweets both overall and within the subset of the most followed authors, suggesting an amplification of discussion surrounding the benefits of telemedicine. Given the relative positivity with which individuals seemed to view telemedicine during the COVID-19 pandemic, shifts in policies stemming from the COVID-19 pandemic that support telemedicine are likely to be well received.