Published on 21.10.20 in Vol 22, No 10 (2020): October
Quantifying Public Interest in Police Reforms by Mining Internet Search Data Following George Floyd’s Death
Background: The death of George Floyd while in police custody has resurfaced serious questions about police conduct that result in the deaths of unarmed persons.
Objective: Data-driven strategies that identify and prioritize the public’s needs may engender a public health response to improve policing. We assessed how internet searches indicative of interest in police reform changed after Mr Floyd’s death.
Methods: We monitored daily Google searches (per 10 million total searches) that included the terms “police” and “reform(s)” (eg, “reform the police,” “best police reforms,” etc) originating from the United States between January 1, 2010, through July 5, 2020. We also monitored searches containing the term “police” with “training,” “union(s),” “militarization,” or “immunity” as markers of interest in the corresponding reform topics.
Results: The 41 days following Mr Floyd’s death corresponded with the greatest number of police “reform(s)” searches ever recorded, with 1,350,000 total searches nationally. Searches increased significantly in all 50 states and Washington DC. By reform topic, nationally there were 1,220,000 total searches for “police” and “union(s)”; 820,000 for “training”; 360,000 for “immunity”; and 72,000 for “militarization.” In terms of searches for all policy topics by state, 33 states searched the most for “training,” 16 for “union(s),” and 2 for “immunity.” States typically in the southeast had fewer queries related to any police reform topic than other states. States that had a greater percentage of votes for President Donald Trump during the 2016 election searched more often for police “union(s)” while states favoring Secretary Hillary Clinton searched more for police “training.”
Conclusions: The United States is at a historical juncture, with record interest in topics related to police reform with variability in search terms across states. Policy makers can respond to searches by considering the policies their constituencies are searching for online, notably police training and unions. Public health leaders can respond by engaging in the subject of policing and advocating for evidence-based policy reforms.
J Med Internet Res 2020;22(10):e22574
The death of George Floyd at the hands of police has resurfaced serious questions about police conduct in the United States that has resulted in 1001 deaths of unarmed persons from 2013 through 2019 . Despite widespread protests, public interest in police reform following George Floyd’s death has not been quantified.
Data-driven strategies that identify and prioritize the needs of the public may engender a public health response to improve policing [, ]. We argue tracking changes in aggregate internet searches is one such strategy, where the content of searches reflects the thoughts and behaviors of the public, the volume of searches reflect their priority, and the location of the search reflects the community [ - ]. This strategy has informed rapid public health responses across many domains, including responses to demand for unproven therapies during the COVID-19 pandemic [ ], demand for HIV testing following Charlie Sheen’s public HIV+ disclosure [ ], interest in sexual harassment prevention and training following the #MeToo movement [ ], and increased suicidal ideation during the initial airing of Netflix’s 13 Reasons Why drama [ ], to name a few. Moreover, once traditional data became available—in some cases years later—these example findings were confirmed [ , ], underscoring the validity of mining search histories. Similarly, public health leaders can survey internet search trends to rapidly detect public interest in and priorities for police reform.
Herein, we assess public interest in police reforms by mining aggregate internet searches and discovering the types of reforms being searched for by the public, both on the national and US state level (including Washington DC). Such reforms include police training, police unions, police militarization, and the qualified immunity doctrine that reduces the liability police officers face for potential misconduct . To translate these trends into potential policy making, we quantified the police reforms most searched for by each state (including DC) and how each state’s political culture reflected by results from the 2016 presidential election explained differences in searches.
We monitored the fraction of daily Google searches (per 10 million searches) that included the terms “police” and “reform(s)” originating from the United States between January 1, 2010, through July 5, 2020. This strategy would capture “reform the police,” “best police reforms,” etc as markers for general interest in policing reforms. We also monitored searches containing the term “police” with “training,” “union(s),” “militarization,” or “immunity.” These searches were selected because they have been frequently cited by experts as avenues for reforms [- ], and represent public prioritization of specific policing reform topics. Search rates were monitored both at the national and state (including DC) level.
Quantifying the Impact of Mr Floyd’s Death
Using historical search rates for each outcome (“police,” “reform[s],” “training,” “union[s],” “militarization,” or “immunity”), we forecasted a counterfactual scenario of expected search rates had Mr Floyd’s death not occurred on May 25, 2020. The expected search rates were forecasted using an autoregressive integrated moving average (ARIMA) model, selected by Hyndman and Khandakar’s algorithm , fit to the historical search rates using all available data January 1, 2010, through May 24, 2020. The forecasted search rates were compared to observed search rates from May 25, 2020, through July 5, 2020, including computing the ratio of observed and expected search rates with bootstrap confidence intervals by day and cumulatively for the entire post period.
Do Search Rates for Specific Police Reform Topics Differ Across States?
First, to assess which states had the greatest number of searches for any policy topic, we estimated the cumulative search rates by state for each policy topic, by summing the fraction of searches per 10 million for the entire post period (May 25, 2020, through July 5, 2020) and ranking states accordingly. Second, to identify which specific policy topics had the greater interest within each state, we estimated each state's total searches for any reform topic, by scaling the cumulative search volumes for each specific policy topic by the total cumulative search volume for all policy topics and ranking states accordingly. For each search query outcome, we contrasted any single state with the mean for all states and compared any two single or group of states by these percentages.
Does Political Culture Impact Searches for Police Reform Policy Topics?
Given the political implications in policy making, we evaluated how political culture impacted searches for specific policy topics, by comparing cumulative search rates by state for each policy topic against their percent vote share for President Donald Trump during the 2016 election. While crude, the presidential percent vote share is often used as an indicator of the political culture of states, reflecting the strength of support or opposition to conservative or liberal policies . For each, we estimated the breakdown of searches for police reform policy topics by states voting for Hillary Clinton or Donald Trump to evaluate any absolute relationships. Secondarily, we evaluated how more support of Trump predicts searches for police reform policy topics by plotting states by their search volume post Mr Floyd’s death and percent vote share for Trump, and estimated the best fitting line including the slope of this line. All analyses were performed using R, version 3.5.1 (The R Project for Statistical Computing).
Internet Searches for Police “Reform(s)”
The 41 days following Mr Floyd’s death corresponded with the greatest number of police “reform(s)” searches ever recorded, with the highest daily fraction of all searches for “reform(s)” eclipsing the past high by over 150-fold (). In absolute terms, this translated into about 1,350,000 total searches for police “reform(s).”
Police “reform(s)” search rates cumulatively increased 3668% (95% CI 2113-6094) above expected levels and began increasing on May 25 with queries first eclipsing the past all-time high 5 days later (May 30) and ultimately setting the new record on June 16. During this time, local protests erupted (May 26), police officer Derek Chauvin was charged with third-degree murder and second-degree manslaughter over Mr Floyd's death (May 29), protests went national (May 31), Mr Floyd's brother testified at a House Judiciary Committee (June 10), Mr Rayshard Brooks’ death was recorded by video while struggling with police (June 12), and protests reached their peak (June 14) . While queries declined, they remained significantly above expected levels through the last day of observation (July 5).
All states and Washington DC had statistically significant increases in search rates for police “reform(s).” By total volume, DC led all states with a total query fraction [QF] of 7300 per 10 million searches during the entire period after Mr Floyd’s death, more than 4.2-fold greater than the average for the states. DC was followed by Alaska (QF=2760, 1.7-fold higher), Vermont (QF=2670, 1.65-fold higher), and Delaware (QF=2640, 1.64-fold higher). The states with the lowest cumulative search rates included Arkansas (QF=783, 55% below the mean), Maine (QF=1011, 41% below the mean), Kentucky (QF=1040, 40% below the mean), and Louisiana (1160, 33% below the mean).
Internet Searches for Specific Police Reform Topics
Searches for specific reform topics also set new national benchmarks (). Searches for “police” and “union(s)” eclipsed the past all-time highs by 4.5-fold, “training” by 4.8-fold, “immunity” by 53-fold, and “militarization” by 34-fold. Statistically, police “immunity” search rates were cumulatively 945% (95% CI 480-6932) higher than expected during the entire post period, followed by “training” (305%; 95% CI 112-1307), “union(s)” (179%; 95% CI 5.8-353), and “militarization” (144%; 95% CI 94-193). This translates into about 1,220,000 total searches for “union(s),” 820,000 for “training,” 360,000 for “immunity,” and 72,000 for “militarization.”
By state, all states including DC had statistically significant increases in searches for specific reform topics after the killing of Mr Floyd, including police “union(s)” (led by Maine, Rhode Island, Wyoming, and West Virginia), “training” (led by Pennsylvania, DC, Minnesota, and Delaware), “immunity” (led by DC, North Dakota, New Mexico, and New Hampshire), and “militarization” (led by DC, New Hampshire, Nevada, and Colorado) (). Search rates were somewhat correlated by state across categories (overall intraclass correlation coefficient=0.31) and the 5 states with the lowest mean search rate across all outcomes after Mr Floyd’s death were Mississippi, Alabama, Tennessee, Texas, and Arkansas (southern states), and 72% (95% CI 64-84) lower than the 5 states with the highest search rate (DC, Pennsylvania, Minnesota, Maine, and Delaware).
Within states, there were typically more searches for police “training” than any other reform topic, followed by police “union(s),” police “immunity,” and police “militarization” (). Still, informative differences emerged in how states favored searching for one specific reform topic over another. In total, 33 states searched more often for police “training” than any other policy, including Alaska, Pennsylvania, Minnesota, and New Jersey, averaging 76% of the total search volume for all 4 reform search topics. In 16 states, police “union(s)” was searched more often than any other topic, including Wyoming, Mississippi, South Dakota, and West Virginia, averaging 66%. Only 2 states searched more for police “immunity” (North Dakota and New Mexico), and no states showed increased searches for police “militarization” compared to all other topics.
Of the states won by President Trump during the 2016 presidential election, 57% searched more for police “union(s),” while 81% of states won by Secretary Clinton searched more for “training.” Increased vote share for Trump predicted a dose-response relationship with searches for police “union(s)” or ”training” (). A 10% increase in vote share for Trump predicted a 4.5% (95% CI 1.3-7.8) increase in cumulative searches for police “union(s)” following Mr Floyd’s death. A similar increase in Trump vote share predicted a 4.3% (95% CI –8.1 to –0.5) decrease in cumulative searches for police “training.” Regardless of a vote preference for Trump or Clinton, states were no more or less likely to search more for police and “immunity” or “militarization” both absolutely and for a marginal change in vote share for Trump.
The United States is at an acute historical juncture with record demand for police reforms sweeping the entire nation. Our results quantify the increasing public interest in police reforms, led by an interest in reforming police training followed by unions, immunity, and militarization nationally. In some states, there is a greater interest in reforming unions or immunity doctrines.
Determining what aspects of police reform to prioritize based on public interest is a challenging problem for elected officials. We argue that monitoring the search queries of a population might clarify the needs and desires of a constituency in real time, especially when little other data is available. The increase in searches related to police reform indicates that the public is actively interested in police reforms, while searches for specific reform topics indicate a desire for leaders to explore these topics. Indeed, many of the reform topics searched for are backed by strong evidence. Restrictions on police unions that potentially advocate for the status quo and hinder reforms and curtailing police immunity that limit criminal and civil liability for police are both effective . On the other hand, some reforms that may be inclusive of searches for “training” are more complex. For example, reforms that implement training on procedural justice or the use of force is associated with less police violence [ ], but racial sensitivity and implicit bias training are not associated with less police violence [ , ], even though unarmed Black individuals are more likely to be shot than unarmed White individuals [ ]. In these cases, a search for training can serve as the basis for leaders to educate the public and seek additional feedback.
The latter raises a serious limitation of our approach; we cannot link search behavior to a policy preference or identify from whom queries originated. Regardless, searches can predict policy priorities on similar public safety issues, and they overcome many of the limitations that often delay traditional data gathering until the point that the data are no longer actionable [, ]. Moreover, our focus is restricted to a few reform topics. Additional reform topics, such as body cameras (which have mixed empirical effect) [ ] or limiting police funding (with unknown impacts and unanticipated effects) [ ] can be studied using our approach. It is important that legislative leaders and policy makers listen to the voices of millions of constituents anonymously searching for solutions and consider policies inclusive of their reform interests as a starting point. Notably, the most high-profile police reform entitled “The Justice Act” proposed by Senator Tim Scott addressed training but did not address police unions, immunity, or militarization. Perhaps amendments to this proposed legislation might consider reforms targeting police unions, given that even in Mr Scott’s home state of South Carolina there were more searches related to police unions than training. Since most policy making occurs among states, decision makers there can rely on our results to identify policies that align with the interests of the public they serve. For instance, in North Dakota or New Mexico, policy makers might consider reforms to police immunity since that reform topic was searched more often than other topics. Extending our work to metropolitan areas can likewise inform more local responses.
While both the American Medical Association  and the American Public Health Association [ ] have spoken out on police violence, the latter calling it a “public health crisis,” health researchers have done little to advance evidence-based police reforms. As of our writing, only 7 studies on PubMed mention “police” and “misconduct” [ ] and 10 mention “police” and “reform(s)” [ ] in their title or abstract. This is not to say there is little scientific work on police reform (there is a tremendous literature [ - ]), but little of it has been conducted from a public health perspective or with the benefit of considering strategies popularized by other public health policy successes. For instance, public health advocates have achieved substantial policy reforms to curb deaths attributable to tobacco use [ , ].
Public health scientists, physicians, and other health professionals must do more. Research on policing is essential to public health practice and aligns with public health’s core mission: prevention. The health community has extensive experience studying relevant areas, including unintended medical errors and systematic bias, and can bring those insights to the topic of policing. Moreover, now is the time for health leaders to insist on evidence-based reforms and support science on police reforms that are responsive to the public's needs.
One way to rapidly leverage science to inform policing is to turn to free and abundant big media data. Researchers could further mine internet search data to answer questions related to police practices. For instance, victims of nonfatal instances of police misconduct might search online for help and these digital tracings could be used to evaluate trends and the geographic distribution of misconduct in almost real time. A similar method is already in practice among health researchers who study drug or device adverse events [, ] and HIV/AIDS, malaria, and tuberculosis in Africa [ ]. Expanding the study of policing to other forms of big media data, including social media, online forums, and news media can similarly provide actionable insights.
Policy changes and public health research in police reform lags while public interest in policing reform is at an all-time high. Research that can help inform policies to improve policing should become a priority among the public health community, with our study serving as a demonstration and call to action. By offering to provide policy makers contemporary insights into policing reform topics their constituents have sought online, we hope that policy makers and elected officials will be inspired to act to prevent unnecessary deaths, like Mr Floyd’s.
This work benefited from intramural support from the Division of Infectious Diseases and Global Public Health and the Center for Data Driven Health at the Qualcomm Institute, both with the University of California San Diego. This work is also supported by a grant from the National Institutes of Health (AI036214).
The funder/sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of their respective employers or funders.
Conflicts of Interest
MD has received consulting fees from Directing Medicine and Good Analytics, which are companies that advise on the use of digital data for public health surveillance. He has also received payments from Sickweather, which uses social media for infectious disease monitoring, and Bloomberg LP. JWA owns equity positions in Directing Medicine, Health Watcher, and Good Analytics, which are companies that advise on the use of digital data for public health surveillance. The remaining authors report no conflicts.
- Mapping Police Violence. URL: https://mappingpoliceviolence.org/ [accessed 2020-06-14]
- Ayers JW, Althouse BM, Dredze M. Could behavioral medicine lead the web data revolution? JAMA 2014 Apr 09;311(14):1399-1400 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009 Mar 27;11(1):e11 [FREE Full text] [CrossRef] [Medline]
- Althouse B, Allem J, Childers M, Dredze M, Ayers J. Population health concerns during the United States' Great Recession. Am J Prev Med 2014 Feb;46(2):166-170. [CrossRef] [Medline]
- Leas EC, Althouse BM, Dredze M, Obradovich N, Fowler JH, Noar SM, et al. Big Data Sensors of Organic Advocacy: The Case of Leonardo DiCaprio and Climate Change. PLoS One 2016;11(8):e0159885 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G, Köhler C. Health-related searches on the Internet. JAMA 2004 Jun 23;291(24):2946. [CrossRef] [Medline]
- Liu M, Caputi TL, Dredze M, Kesselheim AS, Ayers JW. Internet Searches for Unproven COVID-19 Therapies in the United States. JAMA Intern Med 2020 Aug 01;180(8):1116-1118 [FREE Full text] [CrossRef] [Medline]
- Ayers J, Althouse B, Dredze M, Leas E, Noar S. News and Internet Searches About Human Immunodeficiency Virus After Charlie Sheen's Disclosure. JAMA Intern Med 2016 Apr;176(4):552-554. [CrossRef] [Medline]
- Caputi T, Nobles A, Ayers J. Internet Searches for Sexual Harassment and Assault, Reporting, and Training Since the #MeToo Movement. JAMA Intern Med 2019 Feb 01;179(2):258-259 [FREE Full text] [CrossRef] [Medline]
- Ayers J, Althouse B, Leas E, Dredze M, Allem J. Internet Searches for Suicide Following the Release of 13 Reasons Why. JAMA Intern Med 2017 Oct 01;177(10):1527-1529 [FREE Full text] [CrossRef] [Medline]
- Allem J, Leas EC, Caputi TL, Dredze M, Althouse BM, Noar SM, et al. The Charlie Sheen Effect on Rapid In-home Human Immunodeficiency Virus Test Sales. Prev Sci 2017 Jul;18(5):541-544. [CrossRef] [Medline]
- Leas E, Dredze M, Ayers J. Ignoring Data Delays Our Reaction to Emerging Public Health Tragedies Like 13 Reasons Why. JAMA Psychiatry 2019 Sep 25;77(1):103. [CrossRef] [Medline]
- Campaign Zero. URL: https://www.joincampaignzero.org/#vision [accessed 2020-10-08]
- Wood G, Tyler TR, Papachristos AV. Procedural justice training reduces police use of force and complaints against officers. Proc Natl Acad Sci U S A 2020 May 05;117(18):9815-9821. [CrossRef] [Medline]
- Delehanty C, Mewhirter J, Welch R, Wilks J. Militarization and police violence: The case of the 1033 program. 2017 Jun 14;4(2):205316801771288. [CrossRef]
- Rushin S. Police Disciplinary Appeals. University of Pennsylvania Law Review 2019;176(3):545 [FREE Full text]
- Hyndman R, Khandakar Y. Automatic Time Series Forecasting: The forecast Package for R. J Stat Soft 2008;27(3):1-22. [CrossRef]
- Fiorina M, Abrams S, Pope J. Culture War? The Myth of a Polarized America. New York, NY: Pearson Longman; 2005.
- Taylor DB. George Floyd Protests: A Timeline. NY Times. 2020 Jul 10. URL: https://www.nytimes.com/article/george-floyd-protests-timeline.html [accessed 2020-10-08]
- Dharmapala D, McAdams RH, Rappaport J. Collective Bargaining and Police Misconduct: Evidence from Florida. Research Briefs in Economic Policy No. 171 / CATO Institute. 2019 Jul 10. URL: https://www.cato.org/publications/research-briefs-economic-policy/collective-bargaining-police-misconduct-evidence [accessed 2020-10-08]
- Sherman LW. Reducing Fatal Police Shootings as System Crashes: Research, Theory, and Practice. Annu. Rev. Criminol 2018 Jan 13;1(1):421-449. [CrossRef]
- Correll J, Park B, Judd CM, Wittenbrink B. The police officer's dilemma: Using ethnicity to disambiguate potentially threatening individuals. Journal of Personality and Social Psychology 2002;83(6):1314-1329. [CrossRef]
- Ross CT. A Multi-Level Bayesian Analysis of Racial Bias in Police Shootings at the County-Level in the United States, 2011-2014. PLoS One 2015;10(11):e0141854 [FREE Full text] [CrossRef] [Medline]
- Ayers J, Althouse B, Leas E, Alcorn T, Dredze M. Can Big Media Data Revolutionize Gun Violence Prevention? In: Bloomberg Data for Good Exchange Conference. 2016 Presented at: Bloomberg Data for Good Exchange Conference; Sept 25, 2016; New York City, NY p. 25 URL: https://www.researchgate.net/publication/309663843_Can_Big_Media_Data_Revolutionarize_Gun_Violence_Prevention
- Caputi TL, Ayers JW, Dredze M, Suplina N, Burd-Sharps S. Collateral Crises of Gun Preparation and the COVID-19 Pandemic: Infodemiology Study. JMIR Public Health Surveill 2020 May 28;6(2):e19369 [FREE Full text] [CrossRef] [Medline]
- Yokum D, Ravishankar A, Coppock A. A randomized control trial evaluating the effects of police body-worn cameras. Proc Natl Acad Sci U S A 2019 May 21;116(21):10329-10332. [CrossRef] [Medline]
- Weichselbaum S, Lews N. Support For Defunding The Police Department Is Growing. Here’s Why It’s Not A Silver Bullet. The Marshall Project. 2020 Jun 9. URL: https://www.themarshallproject.org/2020/06/09/support-for-defunding-the-police-department-is-growing-here-s-why-it-s-not-a-silver-bullet [accessed 2020-10-08]
- Ehrenfeld J, Harris P. Police Brutality Must Stop. American Medical Association. 2020 May 29. URL: https://www.ama-assn.org/about/leadership/police-brutality-must-stop [accessed 2020-10-08]
- APHA calls out police violence as a public health crisis. American Public Health Association. 2020 Jun 4. URL: https://www.apha.org/news-and-media/news-releases/apha-news-releases/2020/apha-calls-out-police-violence [accessed 2020-10-08]
- ("police"[Title/Abstract] AND "misconduct"[Title/Abstract]). PubMed. 2020. URL: https://pubmed.ncbi.nlm.nih.gov/?term=(%22police%22[Title/Abstract]+AND+%22misconduct%22[Title/Abstract])+&filter=simsearch3.fft&filter=ds1.y_1&sort=date [accessed 2020-10-08]
- ("police"[Title/Abstract] AND "reform"[Title/Abstract]). PubMed. 2020. URL: https://pubmed.ncbi.nlm.nih.gov/?term=%28%22police%22%5BTitle%2FAbstract%5D+AND+%22reform%22%5BTitle%2FAbstract%5D%29&filter=simsearch3.fft&filter=ds1.y_1&sort=date [accessed 2020-10-08]
- Pierson E, Simoiu C, Overgoor J, Corbett-Davies S, Jenson D, Shoemaker A, et al. A large-scale analysis of racial disparities in police stops across the United States. Nat Hum Behav 2020 Jul 4;4(7):736-745. [CrossRef] [Medline]
- Mummolo J. Militarization fails to enhance police safety or reduce crime but may harm police reputation. Proc Natl Acad Sci U S A 2018 Sep 11;115(37):9181-9186. [CrossRef] [Medline]
- Knox D, Mummolo J. Making inferences about racial disparities in police violence. Proc Natl Acad Sci USA 2020 Jan 21;117(3):1261-1262. [CrossRef]
- Healton C. The Tobacco Master Settlement Agreement — Strategic Lessons for Addressing Public Health Problems. N Engl J Med 2018 Sep 13;379(11):997-1000. [CrossRef]
- Frieden TR. Tobacco control progress and potential. JAMA 2014 Jan 08;311(2):133-134. [CrossRef] [Medline]
- Yom-Tov E, Lev-Ran S. Adverse Reactions Associated With Cannabis Consumption as Evident From Search Engine Queries. JMIR Public Health Surveill 2017 Oct 26;3(4):e77 [FREE Full text] [CrossRef] [Medline]
- Yom-Tov E, Gabrilovich E. Postmarket drug surveillance without trial costs: discovery of adverse drug reactions through large-scale analysis of web search queries. J Med Internet Res 2013 Jun 18;15(6):e124 [FREE Full text] [CrossRef] [Medline]
- Abebe R, Hill S, Vaughan JW, Small PM, Schwartz HA. Using Search Queries to Understand Health Information Needs in Africa. Proceedings of the Thirteenth International AAAI Conference on Web and Social Media 2019;13:3-14 [FREE Full text]
|ARIMA: autoregressive integrated moving average|
|QF: query fraction|
Edited by G Eysenbach; submitted 16.07.20; peer-reviewed by YH Lin, JP Allem, A Hamed; comments to author 04.08.20; revised version received 21.08.20; accepted 21.09.20; published 21.10.20
©John W Ayers, Benjamin M Althouse, Adam Poliak, Eric C Leas, Alicia L Nobles, Mark Dredze, Davey Smith. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 21.10.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.