Classifying WHO Followers on Twitter to 6 User Categories
The WHO’s followers on Twitter were classified into 6 categories, including (1) health care professionals, (2) academics, (3) politicians, (4) print and electronic media, (5) legal professionals, and (6) the private sector. This classification was done to investigate if and how different categories on social media react to a public health issue.
We adopted and modified the approach used by Toupin et al [30] to classify Twitter users. The main reason for selecting these categories is that they are influential actors in society whose actions can impact citizens during crisis events as explained in the following paragraphs.
In this study, health care professionals include public health workers in academia and industry sectors, such as doctors and nurses. Health care professionals serve citizens during public health emergencies in various ways, for instance, by warning them against self-medicating [31].
Academics were defined as the people who work in academia (except health care professionals), such as professors, researchers, and students who can inform society through their research and writing about various issues related to the COVID-19 pandemic [32].
Politicians include policy and decision makers in the state and federal governments, such as mayors, congressmen, congresswomen, and senators. Politicians can impact people by, for instance, communicating with them about the concerns raised around vaccine safety [33].
Print and electronic media include those responsible for information dissemination and public awareness, such as journalists, press, news agencies, and publishers. The print and electronic media are powerful sources of information for the public as a crisis unfolds [34]. They can, for example, fact-check the information shared on social media related to COVID-19 [35].
Legal professionals include courts, lawyers, and attorneys who provide legal advice and resources, for instance, for an ideal crisis communication strategy [36]. Additionally, they can ensure the compliance of policies with national laws such as human rights [37].
Private sectors include corporations, incorporated organizations, companies, chief executive officers, and for-profits who serve society by satisfying the needs of citizens. For example, companies can take necessary actions to serve their customers by empowering their employees to operate remotely [38].
The classification of followers into 6 categories was done according to the short biographical profiles of Twitter users. The data sets used in this study can be found online [39]. R software was used to collect and analyze the biographies of Twitter users according to the steps in the following paragraphs.
First, the IDs of the approximately 7.5 million Twitter accounts following the WHO in 2020 were retrieved, and the “username” associated with each ID was collected.
Next, the “description” used in biographies of the accounts (ie, usernames) were collected. Table S12 in Multimedia Appendix 1 provides a list of keywords used to classify and identify the Twitter user categories. The list was generated by searching on the Internet and using websites such as the US Bureau of Labor Statistics [40] that provide a list of job titles in various domains. For instance, the keywords used to identify “academics” were as follows: “lecturer,” “professor,” “phd,” “student,” “ph.d.,” “postdoc,” “postdoctoral,” “doctoral,” “msc,” “master,” “ms,” “bs,” “bachelor,” “undergrad,” “grad,” “graduate,” “undergraduate,” “scientist,” “postgrad,” “faculty,” “chancellor,” “university,” “college,” “school,” “provost,” and “vice-provost.”
To make each category as exclusive as possible and increase the reliability of the classification, general keywords, such as “research,” “researching,” “teams,” “organizations,” “institutes,” “campus,” “professional,” “officer,” “change,” “equity,” and “policy,” were excluded in classifying Twitter users. The preliminary investigation and analysis of user biographies indicated that some keywords such as “chief” or “boss” appeared in several categories. Such keywords were also excluded.
Content Analysis to Identify the WHO’s Agenda
Content analysis was used to identify the main topics discussed by the WHO in the first half of 2020. These topics constitute the WHO’s agenda related to COVID-19 on Twitter. An agenda is the topics presented in the public or media or any other medium at a given time [8].
The tweets posted by the WHO (n=7090) were stratified by months (January 2020 to July 2020) in Microsoft Excel. Using stratified sampling, 10.00% (n=709) of WHO tweets were randomly selected. The sample size for each stratum (ie, month) was proportional to the number of tweets in the stratum [45], as indicated in Table S9 in Multimedia Appendix 1.
The content analysis was carried out in 3 steps: In the first step, called the training phase, the 2 coders familiarized themselves with the content of the tweets. They coded a set of randomly selected tweets together inductively to increase their comfort level with and learn about the content of tweets. This helped them come to the same understanding of how the tweets should be coded [46]. The second phase, called the pilot test, included coders coding 100 of the tweets from a separate representative sample and assessing the intercoder reliability, independently [47], and resolving the discrepancies. The final step included coding the total sample size, independently, which resulted in an intercoder reliability agreement of 85%.
The coding was done by assigning 1 to 3 keywords that described the tweet's content to each tweet. For example, “handwash” and “handrub” were the 2 keywords assigned to a tweet by the WHO [48].
In analyzing the WHO tweets, the context of the tweets was taken into consideration: The keywords that appeared in more than one topic or those not contextually meaningful were removed. For instance, “support” was removed because it appeared in 3 topics. Each topic was identified by a set of exclusive keywords and hashtags (see Table S11 in Multimedia Appendix 1).
This study ensured the keywords within each topic were representative of that category by several rounds of discussions among researchers. We identified 7 topics in the WHO tweets, including “prevention,” “solidarity,” “charity,” “teamwork,” “ill-effect,” “surveillance,” and “credibility.” These topics constitute the WHO’s agenda about COVID-19 in the first half of 2020. The frequency of the 7 topics in the entire WHO data set was calculated. To identify the relationship between the agenda network of the WHO and its followers, we also identified the frequency of each topic for the 6 Twitter user categories. To do this, the frequency of keywords and hashtags within each topic was calculated and summed.
Time Series Modeling (Granger Causality)
Overview
Granger causality has been used in previous agenda-setting studies to examine the relationship between media agenda and public agenda (eg, [51-53]). Granger causality was used in the current study to determine if the changes in 1 variable or time series (a series of data points over time) would impact the changes in another time series [54]. According to Granger [54], Y is said to cause X if the current or lagged values of Y can help to predict the future values of X. It determines whether the future value of a dependent variable can be predicted by the past values of an independent variable. Granger causality determines if there is a correlation between the past values of one variable and the present value of another variable [51].
In this study, Granger causality was used, for instance, to examine if “credibility” as a topic that was promoted by the WHO on Twitter predicted the future values of “credibility” in the tweets posted by the WHO’s followers on Twitter. If the P value of a Granger causality test is less than .05, the independent variable is said to Granger cause or predict the value of the dependent variable [51]. For instance, it can be said Y Granger caused the values of X. However, it is important to note that Granger causality does not mean causation. In the Results and Discussion sections of this study, “influenced” or similar terms such as “affected” will be used instead of “Granger caused.”
To investigate if each time series in the WHO (eg, teamwork) could predict the value of its corresponding time series (ie, teamwork) in any of the Twitter user categories, a vector autoregression (VAR) model was created. Additionally, to investigate if each time series in any of the Twitter user categories (eg, teamwork) could predict the value of its corresponding time series (ie, teamwork) in the WHO, VAR models were created. Overall, 98 VAR models were created in this study.
To perform Granger causality, first, each variable (ie, topic) should be treated as a time series [53]. This study created time series for each of the 7 topics in the WHO and the 6 Twitter user categories, as explained in the following section.
Creating Time Series
To create time series for each topic, the frequency was calculated over time from January 1, 2020, to July 31, 2020, for both the WHO and the 6 categories of WHO followers on Twitter. For each topic, a time series with 182 records was created. Each record contained the frequency of the topic on a specific date. The initial analysis indicated that each time series had many zero number values, which skewed the data. Each zero number represents a lack of data for a given topic on a given date. The highly skewed data violated the normality assumption (see Testing the Residual Normality).
Additionally, having a wide range of values from zero to several hundred led to violating the heteroscedasticity assumption (see Testing the Residual Heteroskedasticity) in many cases. This study tried to resolve the normality and heteroscedasticity issues by aggregating the data to weekly data to reduce zeros. Therefore, for each topic, a time series of 26 weeks was created. Although the data were aggregated to weekly data, the first 2 weeks for most topics still included zero numbers, which again violated the normality assumption in some cases. Removing the first 2 weeks of June 2020 from the analysis resolved the normality issue. Therefore, the analysis was carried out on times series with 24 records (ie, weeks). As an example, the times series for the topics of WHO followers (ie, all 6 Twitter user categories together) are presented in Table S10 in Multimedia Appendix 1.
To perform Granger causality, VAR models should be created first, as explained below.
VAR Model
Overview
A VAR model is used to determine how 2 or more times series influence each other. In a VAR model, each time series is modeled as a linear combination of past values of itself and the past values of other variables [55]. For instance, a VAR model can be used to determine the relationship between the “teamwork” time series in the WHO (TW) and the “teamwork” time series in one of the Twitter user categories (TU) at time (t). Since there are 2 time series, 2 VAR models should be created: 1 for TW and 1 for TU. The VAR model for TW uses the past values of itself (TW) and the past values of the other variable (TU). In its simple form, the VAR model for TW and TU can be as follows, where TWt-1 and TWt-2 are the first and second lags of TW (first variable) and TU t-1 + TO t-2 are the first and second lags of TU (second variable). Each lag is x period ago. For instance, lag one is 1 time period ago or lag two is 2 time periods ago:
TW = TWt-1 + TUt-1 + TWt-2 + TUt-2
TU = TUt-1 + TWt-1 + TUt-2 + TWt-2
In a VAR model, a different number of lags can be considered for each variable. Thus, the optimal number of lags should be selected using a criterion. This study used the Akaike information criterion (AIC) to select the optimal lag for each VAR model, as explained in the following sections.
Lag Selection
The most commonly used criterion in lag selection in VAR models is the AIC [56,57]. The AIC specifies the number of lags to be used in a VAR model [57].
Testing the Stationary Nature
A major assumption underlying the VAR model and Granger causality is that time series must be stationary [54]; otherwise, they should be made stationary using the first or higher differences of the variables. A stationary time series has no systematic trend, meaning that its mean and variance do not change over time [58]. Nonstationary time series lead to incorrect inferences [55]. The Augmented Dickey-Fuller Test was used to test whether the variables were stationary [59].
A time series is stationary if the P value of the Augmented Dickey-Fuller test is less than .05. In this study, all variables in the WHO and the 6 Twitter user categories were nonstationary, which were transformed to become stationary through differencing (first or higher differences of the variables). Differencing is the process in which the differences between consecutive observations of a variable (ie, time series) are computed [60].
Testing the Residual Autocorrelation
A VAR model should be tested to determine if it “provides an adequate description of the data...In time series models, autocorrelation of the residual values is used to determine the goodness of fit of the model. Autocorrelation of the residuals indicates that there is information that has not been accounted for in the model” [61]. The Portmanteau test was used to check the presence of autocorrelation in the models. If the resulting P value in this test is less than .05, autocorrelation exists.
Testing the Residual Normality
Another assumption underlying the VAR model is that its residuals should be normally distributed; otherwise, inferences may be incorrect. The normality of residuals for all models was tested using the multivariate Jarque-Bera test. After performing this test, when the resulting P value is larger than .05, the residuals of VAR model are normal. In 4 cases, BoxCox transformation was used to make the VAR model normal (see Tables S22 and S24 in Multimedia Appendix 1).
Testing the Residual Heteroscedasticity
Another assumption underlying the VAR model is that there should be no heteroscedasticity in residuals. Heteroscedasticity refers to a condition in which the variance of the residual in a regression model varies widely. To test heteroscedasticity, the Autoregressive Conditional Heteroscedasticity-Lagrange Multiplier (ARCH-LM) test proposed by Engle [62] was used. Once the test is performed, if the resulting P value is larger than .05, there is no heteroscedasticity in the data. No heteroscedasticity was observed in the VAR models.