The trial was designed as a parallel group, 2-arm, superiority cluster randomized trial. The clusters—that is, municipalities and city districts—were allocated either to the intervention group or to the control group, where the intervention group received the campaign message and the control group did not receive the campaign message. All municipalities or city districts with at least one person in the target group were included in the study. Randomization was carried out at the cluster level as it was not technically possible to target the campaign to individual persons. The clusters were split evenly between treatments through block randomization with block size two. Before block randomization took place, the list of municipalities and city districts were sorted in accordance with the total number of migrants residing in each area. Partial concealment was ensured as team member 1 created and sorted the list, while team number 2 created a list of pairwise random orders of pairs of A and B, using randomization software [8]. Team number 3 combined these two lists.

Norwegian health authorities have, since the onset of the pandemic, carried out campaigns on social media directed toward the migrant population, using language-based and geographic targeting segments. In December 2020 a “Get tested” campaign was launched, aiming to increase the test rate among migrant populations. Prior to our trial, there were indications of an increased disinterest toward the message, as measured by reach and reactions, and the campaign was paused. To prevent message fatigue [9,10], campaign messages were continuously reviewed, performance measured and customized, and A/B testing conducted—this involved testing different colors, visuals, or creatives to determine which visual received more attention and the time spent on the message. As the “Get tested” campaign had been running for a while, there was by the end of May a small concern that the public had become disengaged or blind to the message; hence, it was decided to roll out new versions. The new versions used short, direct messaging on an A/B-tested colored background. The message remained the same.

We decided that the distribution of the new campaign would be randomized by municipality or city district in order to facilitate an effect evaluation. The content of the relaunched campaign was a sponsored Facebook post from the Norwegian Directorate of Health, distributed in Tigrinya, Polish, Urdu, Somali, Russian, Kurmanji, English, Arabic, Turkish, and Sorani, and focused on the importance of testing for COVID-19 and that testing is free of charge and easy to take. The English message read the following: “COVID-19 tests are free, simple, and completely safe. Get yourself tested to stop the spread of the infection.” Facebook users were free to comment under the post and to share it. Figure 1 shows a screenshot of the post in Polish. Facebook was chosen as a platform in particular because of the high number of Facebook users in Norway in general, and because this platform allowed us to target the position of users with high accuracy.

The campaign was disseminated using targeted Facebook Ads and was launched separately for each municipality or city district and each language. Target groups on Facebook were created by specifying the geographic position and language setting. It was not possible to launch a Facebook campaign when fewer than 100 users matched the targeting criteria. In cases where not enough users matched the language setting, target groups were formed on the basis of behavioral attributes such as “living abroad,” and interest segments specific to the users’ native country. Examples of the latter are as follows: “Syria national football team,” “BBC Urdu,” and “Turkish Kurdistan.” At the time of the trial, Somali and Tigrinya were not available as languages on Facebook. Users who spoke these languages were solely targeted on the basis of behavioral attributes. Lastly, the English campaign was disseminated on the basis of the attribute “living abroad” and the English user language setting.

The campaign was live for a 7-14–day period. The variation in timing was due to the time needed to roll out all the campaigns. Of the 117,436 participants in the intervention group, 52,565 lived in areas where it was possible to target the campaign in accordance with Facebook’s language settings, 58,336 lived in areas where it was only possible to target the campaign on the basis of behavioral attributes or interests, and 4700 lived in areas where it was not possible to launch the campaign, neither based on the participants’ background or language. We cannot know for certain to what extent the intervention group, living in areas where the campaign was rolled out in their language, was actually exposed to the intervention.

The sample included 233,903 persons with a Norwegian national identity number, who are registered as a resident of a Norwegian municipality or city district (in Oslo, Bergen, and Trondheim), aged 18 years or older at the time of assignment, and registered are as having been born in Eritrea, Iraq, Pakistan, Poland, Russia, Somalia, Syria, or Turkey.

We used registry data from the emergency preparedness register for COVID-19 (Beredt C19), which was established by the NIPH in April 2020 [11]. The register contains data on all individuals in the Norwegian Population Register. Individuals were linked across data sources and over time, using an encrypted version of the unique personal identification number provided for every resident of Norway at birth or upon first immigration. In this study, we used individual data from the Norwegian Surveillance System for Communicable Diseases (MSIS) and the National Population Register. MSIS contains data on all polymerase chain reaction (PCR) and rapid tests taken in Norway at official test stations.

Data were also obtained from Facebook on whether it was possible to roll out the campaign, and on the reach, exposure, and frequency of the campaigns that were run in each cluster. These data were anonymized and used only in descriptive analyses, as data from Facebook cannot be linked directly to data on testing.

Data were aggregated into two time periods: baseline data included data collected 14 days to 1 day before the campaign commenced. Outcome data were measured from day 1 to day 14 and from day 1 to day 21 after the campaign started. Day zero was excluded from the analysis. The exclusion of day zero was prespecified in the protocol, and the rationale was that it was unlikely that the effects of the campaign could be observed on the first day.

The outcome was a dichotomous variable, measured at the individual level, indicating whether a person had taken a PCR or antigen test for SARS-CoV-2.

Time invariant covariates included country of birth, sex, and age. These variables were obtained from the National Population Register.

The coefficient of interest was β₁ in regression analysis:

y_j[i]1 = α_j[i] + β₁Treated_i + β₂y_i0 + X_iβ₃ + ε_j[i]t,

where y_j[i]1 indicates whether the individual i in municipality or city district j has taken a COVID-19 PCR or rapid antigen test during the 2 weeks the campaign lasted. y_i0 is the baseline value indicating whether the individual i has taken a COVID-19 test in the 2 weeks prior to campaign commencement. We adjust for the baseline value as is customary in randomized controlled trials (RCTs) [12,13]. X_i is a vector of control variables.

As the outcome is binary, measured at the individual level, and the treatment was assigned at the cluster level, j, multilevel logistic regression analysis was performed to account for intraclass correlation.

The main specification does not take into account that it was not possible to administer the intervention in every municipality or city district assigned to the intervention group. As such, the results represent an “intention to treat” estimate.

The sample size relied on the number of persons aged 18 years and older, born in Eritrea, Pakistan, Poland, Russia, Somalia, Syria, or Turkey, and residing in Norway. The number of languages and countries was determined by the availability of campaign material in the different languages, the distribution of these groups across Norway, and the size of the groups. The language groups were also selected on the grounds of a higher test positivity rate as opposed to the general population. Thus, our sample consisted of all eligible individuals, not a selected sample. Consequently, we performed no prior sample size estimation.

This study does not qualify as health research in the legal sense and hence does not need formal ethics approval. Participant consent was waived because this study was an evaluation of an intervention, using anonymous registry data already collected.

Personal data protection was ensured through the rigorous set up of the Beredt C19 where data on testing and country of birth were linked to the encrypted version of the unique personal identification number. Data from Facebook were collected and aggregated by Mindshare, and could not be linked directly to the registry data.

We conducted this trial to assess the effect of a targeted social media campaign on the rates of testing among Norwegian residents born in Eritrea, Iraq, Pakistan, Poland, Russia, Somalia, Syria, or Turkey. We found a 15.7% relative increase in the proportion taking a COVID-19 test among those who were exposed to the campaign, or an absolute increase of 0.76 percentage points. A distinctive feature of this trial, that strengthens the significance of our findings, was that we were able to assess the effects on actual behavior and not only on intentions or beliefs [14].

We found that targeting in accordance with the users’ language setting yielded a larger effect than that based on behavioral attributes or interest. The content and message remained the same for both targeting approaches. A probable explanation is that the targeting based on users’ language settings reached a higher share of the participants than that based on behavioral attributes or interests.

We are aware of some trials with a similar objective, assessing the effects of social media campaigns on COVID-19–related preventive behavior, COVID-19 knowledge, and intent toward COVID-19 prevention.

We identified 3 RCTs that test the effect of social media campaigns on COVID-19 prevention knowledge. Alsan et al [15] examined how messages read by physicians who varied in age, gender, race, and ethnicity influenced knowledge, beliefs, and practices related to COVID-19. Physician-delivered messages increased knowledge of COVID-19 symptoms and prevention methods for Black and Latinx respondents. The results illustrate that tailoring the message to the targeted groups may increase the effectiveness of the interventions. Similarly, Vandormael et al [16] have conducted a web-based RCT to investigate the effect of a short video on improving COVID-19 prevention knowledge and behavioral intent toward COVID-19 prevention. They concluded the following: “Short, wordless, animated videos, distributed by health authorities via social media, may be an effective pathway for rapid global health communication during health crises” [16]. Lastly, the results of Alatas et al [17] suggest that celebrity endorsement in a social media campaign in Indonesia influenced beliefs about vaccination and knowledge of immunization.

Only Breza et al [18] investigated how social media advertisements affect actual COVID-19 preventive behavior. They showed that short messages recorded by health professionals before the winter holidays in the United States and sent as advertisements to social media users led to reductions in travel (–0.993 percentage points in high-intensity counties, P=.002) and a decrease in COVID-19 infection at the zip code level in the 2-week period starting 5 days post holiday (3.5%, P=.01) [18].

Our results are aligned with those of the aforementioned studies. We demonstrated that targeted advertisements on social media with short, simplified messages can be a valuable tool in the authorities’ toolbox in times of a pandemic. Targeted advertisements can reach audiences that are not easily reached by traditional modes of communication. Furthermore, conveying the message in the mother tongue of the audience can increase trust in the message, although we did not test this specifically.

The effect size was small, which was not unexpected, considering that the COVID-19 prevalence was quite low in the weeks before campaign onset, and that the intervention was a part of a longer ongoing campaign. Norway saw a large increase in test rates among migrants toward the end of 2020 and in early 2021, after the “Get tested” campaign was launched and amid media attention on the high incidence of COVID-19 among migrant groups [19]. It is therefore likely that toward the end of 2021, the target groups were experiencing a certain advertisement fatigue, and that the main message of the campaign was well known both in the control group and the intervention group.

There are some limitations associated with the findings of this study. First, the design of the trial limited the scope and content of the campaign, and the likely external validity. Because we needed to roll out separate campaigns in all areas, it was too arduous to also put out videos or other types of campaign material. In a real-life setting, it would not be necessary to limit users to a very specific geographic unit, the costs of administrating the campaign would be lower, and there would be room for other modes of communication.

Another limitation was that it was not possible to identify the exact effect of actually seeing the posts on testing behavior, as data on reach could not be directly linked with registry data on testing. We know that 82% of the population aged 18 years or older has a Facebook profile [7], but this is only one of many social media platforms. Nevertheless, the total number of users speaking different languages, who were reached, suggests that a large share of the target group was exposed to the campaign.

A related limitation is that the campaign may have had spillover effects on the control group; for example, persons in the intervention group may have shared the post to their Facebook friends and followers. Such organic reach could not be measured, nor was it avoidable. Similarly, the message could have spread from the intervention group to the control group by word of mouth. Spillover effects may have led to an underestimation of the true effect.

Future research could look into effects of communication campaigns on other types of COVID-19 preventive behavior. We also suggest that future campaigns consider using the emergency broadcast capability inherent in cell phone services to deliver targeted advertisements.

Seeing a native-language post on Facebook, in a clear language easily understood manner regardless of health literacy, explaining that testing is simple and can be taken at no cost rendered our target group more likely to take a COVID-19 test. This study demonstrated that targeted social media advertisements sponsored by health authorities can influence individual behavior in an infection control–friendly direction during a pandemic.