This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Over 1 million people in the United States have died of COVID-19. In response to this public health crisis, the US Department of Health and Human Services launched the
We aimed to address this gap by assessing the association between the
A nationally representative sample of 3642 adults recruited from a US probability panel was surveyed over 3 waves (wave 1: January to February 2021; wave 2: May to June 2021; and wave 3: September to November 2021) regarding COVID-19 vaccination, vaccine confidence, and sociodemographics. Survey data were merged with weekly paid digital campaign impressions delivered to each respondent’s media market (designated market area [DMA]) during that period. The unit of analysis was the survey respondent–broadcast week, with respondents nested by DMA. Data were analyzed using a multilevel logit model with varying intercepts by DMA and time-fixed effects.
The
Results from this study provide initial evidence of the
The COVID-19 pandemic has led to more than 104 million COVID-19 cases and over 1 million COVID-19 deaths in the United States as of April 3, 2023 [
Vaccine hesitancy is a critical barrier to vaccine uptake [
Public education campaigns, which reach and engage large population segments through a mix of media channels, have demonstrated a measurable impact on a range of health behaviors [
The
Between April 5 and September 26, 2021, according to Nielsen Digital and Total Ad Ratings (see
To date, there have been no published evaluations of the impact of this campaign on COVID-19 vaccine uptake. This study is the first to assess the association between digital campaign media dose—an under-studied avenue for public education campaign dissemination—and an individual’s likelihood of receiving their first COVID-19 vaccination dose.
To evaluate the potential association between the campaign and vaccine uptake, we used individual-level survey data and market-level campaign media dose data. Digital campaign media dose refers to the aggregation of all digital ads that were placed in a DMA at a given time. The individual-level data were derived from the COVID-19 Attitudes and Beliefs Survey (CABS), a nationally representative, probability-based longitudinal survey of US adults (aged 18+ years) administered every 4 months to the same individuals through the AmeriSpeak probability-based research panel of the National Opinion Research Center (NORC) [
We sought institutional review board (IRB) approval for this study from the Biomedical Research Alliance of New York, an external IRB service accredited by the Association for the Accreditation of Human Research Protection Programs. The study protocol and materials were reviewed and approved by Biomedical Research Alliance of New York’s social and behavioral IRB (Federalwide Assurance FWA00000337, protocol 20-077-821).
Although all respondents provided consent as part of their registration into their associated panel, we ensured that all qualifying respondents provided informed consent to participate in the study. The consent language was available on the web, programmed into the final part of the screener. After screening respondents, we directed those eligible (ie, respondents who did not screen out) to read the consent language. If they decided to participate, eligible respondents electronically provided consent and were directed to the web-based survey. Although this study presented minimal risk of harm to subjects, all respondents were informed at the beginning of the survey that any questions that make them feel uncomfortable may be skipped or ignored. We included links to mental health resources for respondents to access if they experienced any distress from participating in the study.
To ensure respondent confidentiality, (1) data transfer was conducted via a secure, password-protected site; (2) all screening-related information was not tied to any personal identifiable information, but identified and matched by the assigned unique ID; (3) data sets and reports did not contain any personal identifiable information; and (4) respondents were not tied to individual responses, and any data used in reporting were not be attributed to specific respondents. Data were tightly controlled behind firewalls with password-protected access by senior researchers. All final data were stored in a secure environment that does not have access to the internet and requires a separate access code by researchers. Researchers were trained to never export data from this secure server.
Respondents who decided to participate were offered US $10 in the first wave of the CABS and US $18 for each subsequent wave of the survey.
The dependent variable was dichotomous, indicating whether a respondent reported receiving the first dose of a COVID-19 vaccination in each broadcast week. The unit of analysis was the respondent-broadcast week; we used broadcast weeks, which run from Monday to Sunday, because that is how advertising is purchased. Within the data set, there was an observation for each CABS respondent in each broadcast week starting the week of November 30, 2020, as this date marks the beginning of the first broadcast week in which a vaccine was publicly available. If a respondent did not report having been vaccinated in a broadcast week, then they were included as an observation in the subsequent broadcast week. If a respondent reported having been vaccinated in a broadcast week, then they were not included as an observation in the subsequent broadcast week. Some respondents (n=241) reported vaccination dates that occurred before the date of the US Food and Drug Administration (FDA) emergency use authorization (EUA). Under the assumption that these individuals misstated the year of vaccination, it was changed from 2020 to 2021 in these instances. As a robustness test, we conducted analyses in which individuals who reported a vaccination date before the FDA EUA were dropped. As the results were similar, we retained them in the analysis.
The independent variable was paid campaign digital media dose, representing the change in the total number of site direct, programmatic, and social media advertisement impressions (impressions are the digital publishers’ estimates of the number of times an advertisement is seen or heard) in a DMA (a DMA region is a group of counties and zip codes that form an exclusive geographic area in which the home market television stations are the predominant stations in terms of total hours viewed; DMA is a proprietary construct of the Nielsen Company) per 100,000 people between
To account for the potential influence of factors exogenous to the campaign that could still be correlated with changes in media dose, analyses controlled for the change in weekly COVID-19 cases and deaths and the change in weekly cable news COVID-19 coverage by DMA between
We expected that individuals may have predispositions that may influence the effectiveness of the campaign and their likelihood to get a vaccination, so we controlled for respondent vaccine confidence (ie, a respondent’s reported vaccine uptake or likelihood that they will get vaccinated against COVID-19) as reported in CABS wave 1. Details about COVID-19 cases and deaths data, COVID-19 cable news coverage data, sociodemographic variables, and vaccine confidence, including a discussion of the coding of these variables, are provided in
Before conducting analyses, we examined the independent and dependent variable distributions to inform our analytic approach. The earliest a respondent reported receiving the first dose of a COVID-19 vaccine was December 2, 2020 (see
The change in digital media dose by DMA, the main independent variable, ranged from –155,716.4 to 117,041.3, with a mean of 520.6 (SD 14, 281.69). Table S1 in
To assess the relationship between digital media dose and first-dose COVID-19 vaccination, we estimated a series of multilevel logistic regression models, with varying intercepts by DMA to account for the nesting of respondents within DMAs. The SEs of these models are clustered by DMA. The intraclass correlation coefficient for the main model was 0.03, indicating that about 3% of the variance in the outcome variable varies across DMAs.
We estimated 4 regression models in a stepwise manner. Model 1 (baseline model) estimates the relationship between change in digital media dose between
Histogram of reported first dose of COVID-19 vaccination dates, United States, December 1, 2020, to November 7, 2021.
Results for model 1 show a positive and statistically significant relationship between the weekly change in digital impressions and the likelihood of first-dose vaccination (β=.000014; Z=3.22;
There were no substantive differences between models 1 and 2 in the effects of change in digital media dose on the likelihood of first-dose COVID‑19 vaccination; the effect on the likelihood of first-dose vaccination continued to be positive (β=.000014; Z=3.16;
Relationship between digital advertising media dose and the likelihood of vaccination, United States, December 1, 2020, to November 7, 2021a.
|
Model 1 | Model 2 | Model 3 | Model 4 | ||||||||||||||
|
Value | Value | Value | Value | ||||||||||||||
Δ HHSb digital impressions, β (SE) | 0.000014 (0.000004) | .001 | 0.000014 (0.000004) | .002 | 0.000013 (0.000004) | .002 | 0.000014 (0.0000004) | .002 | ||||||||||
|
||||||||||||||||||
|
Δ COVID-19 cases | —c | — | –0.0006 (0.0003) | .047 | –0.0006 (0.0003) | .04 | –0.0006 (0.0003) | .051 | |||||||||
|
Δ COVID-19 deaths | — | — | –0.0105 (0.0090) | .24 | –0.0069 (0.0085) | .42 | –0.0073 (0.0084) | .38 | |||||||||
|
Δ COVID-19 cable news coverage | — | — | 0.1720 (0.7057) | .81 | 0.1670 (0.7201) | .81 | 0.1515 (0.7297) | .84 | |||||||||
|
||||||||||||||||||
|
Income | — | — | — | — | 0.169 (0.022) | <.001 | 0.115 (0.028) | <.001 | |||||||||
|
Female sex | — | — | — | — | –0.114 (0.051) | .03 | –0.010 (0.06) | .88 | |||||||||
|
Age | — | — | — | — | 0.456 (0.04) | <.001 | 0.378 (0.04) | <.001 | |||||||||
|
Education | — | — | — | — | 0.219 (0.03) | <.001 | 0.169 (0.04) | <.001 | |||||||||
|
Essential worker status | — | — | — | — | –0.101 (0.07) | .13 | –0.060 (0.08) | .47 | |||||||||
|
Political ideology | — | — | — | — | –0.448 (0.036) | <.001 | –0.245 (0.043) | <.001 | |||||||||
|
Preexisting health condition | — | — | — | — | 0.191 (0.055) | .001 | 0.099 (0.055) | .08 | |||||||||
|
Rurality | — | — | — | — | –0.118 (0.046) | .01 | –0.095 (0.049) | .05 | |||||||||
|
Black or African American race | — | — | — | — | –0.170 (0.085) | .04 | 0.035 (0.076) | .64 | |||||||||
|
Hispanic/Latino ethnicity | — | — | — | — | 0.154 (0.073) | .04 | 0.177 (0.075) | .02 | |||||||||
|
||||||||||||||||||
|
Wave 1 Vaccine Confidence | — | — | — | — | — | — | 1.070 (0.058) | <.001 | |||||||||
|
||||||||||||||||||
|
Constant, β (SE) | –3.5199 (1.068) | .001 | –2.240 (4.449) | .62 | –2.637 (4.555) | .56 | –5.0876 (4.639) | .27 | |||||||||
|
DMAd variance, β (SE) | 0.103 (0.024) | — | 0.100 (0.024) | — | 0.090 (0.022) | — | 0.111 (0.029) | — | |||||||||
|
Observations, n | 76,128 | — | 76,128 | — | 76,128 | — | 76,128 | — | |||||||||
|
DMAs, n | 204 | — | 204 | — | 204 | — | 204 | — |
aThe dependent variable is a dichotomous measure of whether a respondent received the first dose of a COVID-19 vaccine in each week.
bHHS: Department of Health and Human Services.
cNot applicable.
dDMA: designated market area.
The relationship between change in digital media dose and the likelihood of first-dose vaccination was positive and statistically significant for both models 3 and 4 (βmodel3=.000013; Z=3.12;
Taken together, models 1-4 consistently indicated that an increase in the number of digital impressions in a DMA between
To examine the substantive relationship between the change in digital campaign impressions and the likelihood of first-dose COVID-19 vaccination, we estimated the expected probability of first-dose vaccination across levels of weekly change in digital impressions while holding all other variables at their means. More than 95% of all observations fell between –30,000 and 30,000 impressions, although this variable ranged from about –155,000 to a high of about 117,000 in some markets (
The distribution of change in digital impressions, United States, April 1, 2021, to November 7, 2021.
Average change in weekly digital impressions on the likelihood of individual first-dose vaccination, United States, December 1, 2020, to November 7, 2021.
When digital impressions in a DMA decreased by 30,000 between
This study assessed the relationship between paid campaign digital media and the likelihood of COVID-19 vaccination in a representative sample of US adults. Results demonstrate a positive and significant relationship between the weekly change in digital impressions and the likelihood of first-dose vaccination, providing initial evidence that the digital campaign has been effective in increasing COVID-19 vaccination among US adults. This association remained statistically significant after controlling for a series of covariates, including COVID-19 cases and deaths as well as respondents’ sociodemographic characteristics and baseline vaccine confidence, indicating that results are robust to the inclusion of other factors. It is possible that the association is a factor of both getting people who would otherwise not be vaccinated to do so and shortening the time to vaccination, both of which are of substantial importance during a pandemic. Future research could use event-history modeling to explore these possibilities.
Results indicate that when change in digital impression exposure is held at 0, older respondents compared to younger respondents, those with higher incomes compared to those with lower incomes, and those with higher education compared to those with lower levels of education were significantly more likely to receive first-dose COVID-19 vaccination. These findings, which are independent of the campaign’s effects, align with recent research demonstrating that willingness to vaccinate is higher among individuals aged 65 years and older compared to younger groups [
Importantly, much of the extant literature examines longer-term effects (eg, recalled campaign exposure or campaign impressions aggregated over longer periods of time) on behavior change [
This study’s results reflect a discrete period and single media channel and may not reflect the influence of the campaign on the likelihood of vaccination during other time periods or channels through which the campaign has been disseminated (eg, print and radio). The recalled date of first-dose vaccination was subject to recall bias and may not reflect respondents’ actual date of vaccination. All impressions data were aggregated by DMA; however, the dependent variable was provided at the respondent level. Weekly changes in media dose by DMA functioned as a measure of probable dose, exogenous to our survey data, but does not represent confirmed campaign exposure among respondents. It is possible that weighting methodologies could influence findings; however, sensitivity checks found little change in the association (
Although our models included potential influencing factors for vaccine uptake, the variable list was not exhaustive, and analyses may have been subject to the influence of unmeasured confounders. For more than 2 years, US adults have been exposed to information and conversations about COVID-19 vaccination from government sources (eg, federal agencies and state and municipal health departments), health care representatives (eg, health care professionals and pharmaceutical companies), community-based organizations, and friends and family. Concurrently, many government, travel, and employer vaccination mandates and policies were implemented during the study period. It is possible that first-dose vaccination in our study sample was influenced by one or several other factors not included in our models. Further, the change in the campaign’s digital dose may have differential effects based on geography, with the campaign being more successful in certain regions of the country. This may be an interesting area for future research.
The COVID-19 pandemic represents one of the largest public health crises of our era [
This study’s findings show that the HHS COVID-19 public education campaign was associated with a greater likelihood of individual vaccination in any given week during the period of April 1 to November 7, 2021. People who reported living in areas with more digital campaign impressions were more likely to be vaccinated, as increasing digital impressions from –30,000 to 30,000 in a given week more than doubled (125% increase) the likelihood of being vaccinated. These findings indicate that, similar to public education campaign influence on other health behaviors [
Additional methods, Nielsen Digital Ad Ratings, and supplementary tables.
COVID-19 Attitudes and Beliefs Survey
designated market area
emergency use authorization
Food and Drug Administration
Department of Health and Human Services
institutional review board
National Opinion Research Center
This work was supported by the US Department of Health and Human Services (HHS) using National Institutes of Health (NIH) contract #75N98019D00007 under orders 75N98022F00001, 75N98021F00001, and 75N98020F00001. The authors gratefully acknowledge our colleagues at the HHS Office of the Assistant Secretary for Public Affairs (ASPA); the HHS Office of the Assistant Secretary for Planning and Evaluation (ASPE), especially Trinidad Beleche, Nicholas Holtkamp, and Lok Wong Samson; the Centers for Disease Control and Prevention (CDC), especially Lynn Sokler; and the Fors Marsh–led team of agencies contributing to the implementation and evaluation of this campaign. We thank the thousands of research respondents who made this study possible.
This publication represents the views of the authors and does not represent US HHS position or policy.
Data used for this analysis are currently not available due to permissions.
None declared.