This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The Affordable Care Act (ACA), often called “Obamacare,” is a controversial law that has been implemented gradually since its enactment in 2010. Polls have consistently shown that public opinion of the ACA is quite negative.
The aim of our study was to examine the extent to which Twitter data can be used to measure public opinion of the ACA over time.
We prospectively collected a 10% random sample of daily tweets (approximately 52 million since July 2011) using Twitter’s streaming application programming interface (API) from July 10, 2011 to July 31, 2015. Using a list of key terms and ACA-specific hashtags, we identified tweets about the ACA and examined the overall volume of tweets about the ACA in relation to key ACA events. We applied standard text sentiment analysis to assign each ACA tweet a measure of positivity or negativity and compared overall sentiment from Twitter with results from the Kaiser Family Foundation health tracking poll.
Public opinion on Twitter (measured via sentiment analysis) was slightly more favorable than public opinion measured by the Kaiser poll (approximately 50% vs 40%, respectively) but trends over time in both favorable and unfavorable views were similar in both sources. The Twitter-based measures of opinion as well as the Kaiser poll changed very little over time: correlation coefficients for favorable and unfavorable public opinion were .43 and .37, respectively. However, we found substantial spikes in the volume of ACA-related tweets in response to key events in the law’s implementation, such as the first open enrollment period in October 2013 and the Supreme Court decision in June 2012.
Twitter may be useful for tracking public opinion of health care reform as it appears to be comparable with conventional polling results. Moreover, in contrast with conventional polling, the overall amount of tweets also provides a potential indication of public interest of a particular issue at any point in time.
Americans have strong opinions about health care reform. Polls of the general public consistently indicate that less than half of Americans support the Affordable Care Act (ACA) [
Timeline of key events related to the implementation of the Affordable Care Act.
Date | Event |
March 23, 2010 | ACAa signed into law by President Obama. Key coverage provisions—Medicaid expansion and health insurance exchanges—are scheduled to take effect in January 2014. Multiple lawsuits challenging different provisions of the law are filed shortly after its enactment. |
2010-2011 | Early ACA provisions are implemented, including consumer protections (eg, prohibitions on annual and lifetime caps on coverage) and the requirement that employer-sponsored plans must offer coverage for dependent children up to age 26 years. Most of these take effect as private plans were renewed; as a result, they do not have a single “headline” date for implementation. |
December 19, 2011 | The SCOTUSb announces it will hear oral arguments in NFIBc versus Sebelius, challenging the constitutionality of two key ACA provisions: the requirement that all individuals have coverage (the “individual mandate”) and the expansion of Medicaid to all individuals with incomes below 138% of poverty. |
March 26-28, 2012 | The SCOTUS hears oral arguments in NFIB versus Sebelius, generating tremendous speculation. |
June 28, 2012 | The SCOTUS rules in NFIB versus Sebelius. The individual mandate is affirmed whereas the Medicaid expansion is effectively rendered optional for states: a mixed decision, but on balance regarded as a win for the ACA. |
November 6, 2012 | President Barack Obama reelected. |
October 1, 2013 | The first open enrollment period begins for private health insurance exchanges; the federal exchange website healthcare.gov fails to work properly, generating negative publicity. |
November 26, 2013 | The SCOTUS announces it will hear oral arguments in Burwell versus Hobby Lobby, challenging a private employer’s refusal on religious grounds to provide full insurance coverage for contraception. |
January 1, 2014 | Expanded coverage through health insurance exchanges starts. |
March 31, 2014 | The first open enrollment period ends. |
March 25, 2014 | The SCOTUS announces it will hear oral arguments in Burwell versus Hobby Lobby, which is about whether corporations owned by religious families can refuse to comply with an ACA requirement that their health insurance must fully cover contraception for female workers. |
June 30, 2014 | The SCOTUS rules in favor of the corporations in Burwell versus Hobby Lobby (a blow to the ACA). |
November 8, 2014 | The SCOTUS announces it will hear oral arguments in King versus Burwell, which challenges the payment of federal subsidies for health insurance in states that rely on healthcare.gov (a majority of states). |
November 15, 2014 | The second open enrollment period begins for private health insurance exchanges; healthcare.gov works as intended. |
February 15, 2015 | The second open enrollment period ends for private health insurance exchanges. |
March 4, 2015 | The SCOTUS hears oral arguments in King versus Burwell. |
June 25, 2015 | The SCOTUS Court rules in favor of the Obama administration in King versus Burwell. |
aACA: Affordable Care Act.
bSCOTUS: Supreme Court of the United States.
cNFIM: National Federation of Independent Business.
Public opinion may have briefly dipped or risen immediately after these key events [
Monitoring public response to new laws and regulations, such as those included in the ACA, is of considerable interest to health policymakers, government agencies, and the media. Traditionally, measuring public response has relied on expensive and time-consuming surveys administered by polling agencies including the Pew Research Center and the Kaiser Family Foundation. Changes in technology introduce new opportunities for tracking public response. One particularly rich source of data is Twitter. Twitter has been used to study natural disasters [
The aim of this paper was to examine the extent to which Twitter data can be used to measure public response to the rollout of the ACA. Our specific research questions were: (1) To what extent can ACA-related tweets be accurately identified? (2) Does the overall volume of ACA-related tweets respond to key events in the implementation of ACA? (3) Is there an association between public opinion (ie, favorable vs unfavorable) measured using ACA-related tweets and conventional polling data from the Kaiser Family Foundation health tracking poll? and (4) What are common words used in favorable versus unfavorable ACA-related tweets?
We examined the extent to which Twitter data can be used to measure public response to the ACA over time. To do so, we identified relevant tweets over a 6-year time period, examined them, and compared the ACA-related tweet sentiment with conventional polling data of public opinion. This study used publicly available data for all analyses and was deemed to be exempt from institutional board review.
We prospectively collected a 10% random sample of daily tweets (approximately 52 million since July 2011) using Twitter’s streaming API (ie, the “Twitter Gardenhose”) from July 10, 2011 to July 31, 2015. All analyses were restricted to English-language tweets.
To identify tweets about the ACA in this sample, we developed a list of key search terms. From the ACA Wikipedia page [
We also used Twitter hashtags to expand our identification of tweets about the ACA (
To check the validity of this method for identifying tweets related to the ACA we pulled a random sample of 1000 tweets. Two separate members of the research teach reviewed each tweet to determine if it was indeed relevant to the ACA. Thirty-seven of these tweets were not in fact ACA-related (in several the tweet in question used the term ACA as an abbreviation for “acapella” and the tweet was related to singing;
Terms used in tweets
• Affordable Care Cct or ACA
• Healthcare insurance exchanges
• Healthcare reform act or bill
• Healthcare insurance act or bill
• Obamacare
• Patient Protection and Affordable Care Act or PPACA
Hashtags
• #ACA
• #aca
• #Obamacare
• #ObamaCare
• #obamacare
Selected examples of relevant and nonrelevant Affordable Care Act-related tweets.
Type | Examples |
Relevant favorable ACAa tweets | “Finally, my two favorite things come together: online shopping and buying health insurance.” |
“In response to Obamacare, nearly 1 in 3 health facilities are adding doctors.” | |
“The GOP Is Terrified Obamacare Could Be a Success.” | |
“Thanks to the ACA, Over 5800 Californians with Pre-Existing Conditions Now Getting Care.” | |
“Obamacare winning one step at a time, sometimes take double steps. Today, good news for people with heart disease.” | |
Relevant unfavorable ACA tweets | “Dems Throwing Granny Off the Cliff: Obamacare Cuts Medicare, Seniors Losing Doctors.” |
“4 Years Later ObamaCare Still a Crime Against Democracy That The American People Will Never Accept.” | |
“Obamacare: Biggest Job-Killing TAX in US History!” | |
“The people of America have no concept at this point as to just how miserable Obamacare is going to make individual lives.” | |
“Weird new error screen for Obamacare.” | |
Nonrelevant ACA tweets | “You’re one of those acapella girls, I’m one of those acapella boys, and we’re gonna have aca-children.” |
“Who’s watching the ACA’s?!” | |
“Thank you guys so much for last night. The aca awards were a blast. Thanks for making 2013 unbelievable.” |
aACA: Affordable Care Act or acapella.
We used standard text sentiment analysis to assign each ACA tweet a measure of positive to negative sentiment. Text sentiment analysis uses a lexicon of words each with previously assigned numeric measures of emotion (ranging from negative to positive, ie, −1.0 to +1.0). In this study, tweet sentiment was measured using labMT, a lexicon developed by Dodds et al based on human review of terms from language used in Twitter, Google Books, music lyrics, and the New York Times) [
After tweets were processed to remove words that do not convey specific content (such as “a” or “the”), the assigned scores for words in a given text were summed up to arrive at an overall score of the sentiment. Tweets with a sentiment score greater than zero were coded favorable while those with a score less than zero were coded unfavorable.
Since the enactment of the ACA in March 2010, the Kaiser Family Foundation’s health tracking poll has been conducted monthly to evaluate the public views of the ACA [
From the Kaiser poll data, we determined the percent of respondents who reported being favorable versus unfavorable toward the ACA by month. Kaiser data were not available for 5 months (December 2012, January 2013, May 2013, July 2013, and August 2014).
Descriptively, we sought to examine the influence of key events regarding the ACA implementation on public response. Therefore, we identified the following historical events that took place during the study period (see
Across calendar months, we used Spearman correlation to evaluate for associations between public response measured using ACA tweets and the Kaiser poll. For instance, we compared the percentage of unfavorable ACA tweets per month with the percentage of Kaiser poll respondents who were unfavorable toward the ACA. As young adults tend to use Twitter more than older adults, we also examined correlations stratified by the age of Kaiser poll respondents [
To determine whether Americans were referring to the ACA as “Obamacare” more or less over time, we show the volume of ACA-related tweets that do and do not contain this term. For all analyses, we used R statistical software (R Foundation for Statistical Computing, Vienna, Austria). A 2-sided
We also performed a subanalysis to test the robustness of the associations we observed to determine whether tweets from political and special interest groups impacted our results. To do so, we reanalyzed associations after excluding the 310,862 clearly political ACA tweets that included hashtags such as #gop (Grand Old Party), #teaparty, #p2 (Progressive 2.0), #PJNET (Patriot Journalist Network), #tlot (Top Libertarians on Twitter), #ccot (Christian Conservatives on Twitter), and #tcot (Top Conservatives on Twitter).
Finally, to provide some insight into the content of favorable versus unfavorable ACA tweets, we calculated the most frequently used other words (ie, not used to identify the tweets as ACA-related) and displayed these using word clouds.
In spite of these differences our Twitter-based measure of public opinion track results of the Kaiser poll quite well over time. The correlation coefficient between percentage of unfavorable ACA tweets and Kaiser respondents was .43,
Favorable (A and B) versus unfavorable (C and D) public response to the Affordable Care Act using Tweets compared to results from the Kaiser Poll.
Because Twitter users are likely to be younger than average, we also compared the percentage of favorable and unfavorable tweets according to Kaiser respondent age group. The strongest correlation was for unfavorable public response among Kaiser respondents between 18 and 29 years of age—a correlation coefficient of .47,
Whereas public opinion may not change (much) in response to significant events in the ACA’s history, the volume of ACA-related tweets certainly does (
Correlation between percentage of favorable (or unfavorable) tweets and percentage of favorable (or unfavorable) Kaiser poll respondents about the Affordable Care Act.
Kaiser respondents | Spearman correlation coefficient ( |
|||
Favorable | Unfavorable | |||
All | .37 (.02) | .43 (.01) | ||
By age category | ||||
18-29 years | .14 (.36) | .47 (.01) | ||
30-49 years | .41 (.01) | .40 (.01) | ||
50-64 years | .21 (.21) | .12 (.43) | ||
65+ years | .22 (.17) | .08 (.59) |
Total number of Affordable Care Act-related Tweets per month from July 2011 to January 2015.
Finally, in order to shed some light on the content of favorable and unfavorable ACA-related tweets, we tabulated the words most commonly used in each type of tweet, excluding the search terms in
Common words used in favorable (A) versus unfavorable (B) Affordable Care Act-related Tweets.
To our knowledge this is the first study to compare Twitter-based measures of public opinion regarding the ACA to traditional polling results. Overall, we found evidence that Twitter data can be effectively leveraged to estimate public opinion, including the response (or lack of response) of opinion to specific events such as health care reform. Trends in the overall public response measured by sentiment of tweets paralleled the results of Kaiser poll, and the levels of favorable and unfavorable response were quite similar over time. Not surprisingly, public response to the ACA on Twitter correlated most highly with polling data for younger adults —the age group most likely to use social media platforms [
The most striking finding may be that our Twitter-based measures of ACA public response exhibit the same remarkable stability over time that characterizes results from the Kaiser poll. One of the central puzzles about public opinion toward the ACA—why are opinions changing so little over time, even as major components of the law have been implemented and provided health insurance coverage to millions of Americans?—is just as pronounced in the immediate-response world of social media as in the more staid world of traditional opinion polling. Public opinion on Twitter toward the ACA could be more volatile, or more malleable, than opinions measured by the Kaiser poll—but they aren’t. The lack of significant change in favorable or unfavorable views toward the ACA over time does not mean people aren’t paying attention. On the contrary, they are not only paying attention, they are also expressing opinions in response to key events as we identified large changes in the volume of ACA tweets over time. The most striking spike in ACA tweets was in response to the first open enrollment period in October 2013 (during which the exchange enrollment website healthcare.gov failed to function properly). These large changes in volume, coupled with the lack of concomitant change in the favorable or unfavorable nature of the sentiments being addressed, echoes the thesis first advanced by Iyengar and Kinder that news may not so much change opinions as change how they are expressed [
There is growing use of Twitter to quantify public response. For instance, the sentiment expressed in tweets detected using either automated or manual annotation has been used to measure public response to vaccinations [
The chief limitation of Twitter data is that Twitter users are, by definition, not representative of the general population. Any ways in which Twitter users are different from the typical American—for example, being younger, more tech-savvy, or having a different political orientation—could bias our Twitter-based estimate of ACA sentiment, if these underlying differences also affect attitudes toward the ACA. This is the reason why our analysis begins by comparing our estimates with estimates from the nationally-representative Kaiser poll. Other limitations include the fact that our algorithms for identifying ACA-related tweets and for encoding the sentiments they contain could introduce systematic bias. Whereas these methods represent the current state of the art in social media analysis, this relatively young field is evolving rapidly and subsequent methodological refinements may improve on the approach we use here.
In this study we found some evidence that Twitter may be useful for tracking public opinion regarding US health care reform as it appears to be comparable with conventional polling results. Similar to previous studies that used Twitter to measure public response, we found large changes in the amount of tweets in relation to key events; yet, during these time periods public opinion appeared to changed very little. Thus, the overall amount of tweets may also provide a potential indication of general public interest of a particular issue at any point in time. Whereas use of social media data for tracking public opinion is not without limitations, it is inexpensive, immediate, and can offer contextual insights beyond that of conventional polling.
Affordable Care Act
application programming interface
This work was supported by the National Institutes of Health (NIH) [K01 AT006162 (Davis) and K01 AG034232 (Levy)]. The NIH had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript. The views expressed are those of the authors and do not necessarily reflect those of the NIH.
None declared.