This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The prevalence of social media use among youth and young adults suggests it is an appropriate platform for study recruitment from this population. Previous studies have examined the use of social media for recruitment, but few have compared platforms, and none, to our knowledge, have attempted to recruit cigarillo users.
The purpose of this study was to examine the effectiveness of different social media platforms and advertisement images for recruiting cigarillo users aged 14-28 years to complete a cigarillo use survey.
We obtained objective data for advertisement impressions for a 39-week social media recruitment campaign. Advertisements were targeted to users based on their age, geography, and interests. Effectiveness was defined as the percentage of approved surveys per advertising impression. Chi-square tests were performed to compare the effectiveness of different advertisement images and platforms.
Valid survey completers (n=1089) were predominately older (25-28 years old, n=839, 77%). Of the 1089 survey completers, 568 (52%) identified as male, 335 (31%) as African American, and 196 (18%) as Hispanic. Advertisements delivered via Facebook/Instagram were more effective than Twitter; 311/1,027,738 (0.03%) vs 661/2,998,715 (0.02%);
The campaign was effective in recruiting a diverse sample representative of relevant racial/ethnic categories. Advertisements on Facebook were more effective than Twitter. Advertisements that featured an image of a cigarillo were consistently the most effective and should be considered by others recruiting cigarillo users via social media.
Recent data on smoking prevalence indicates that a substantial proportion of youth in the US smoke cigar products, including cigarillos [
Recruiting cigarillo users for survey research presents unique challenges using traditional methods. Youth and young adults are less likely to respond to survey recruitment via telephone and change their residency often, making mail-based recruitment less effective [
Web-based recruitment using social media may be particularly appropriate in reaching youth and young adult cigarillo users [
A growing body of research has examined the use of social media advertisements for tobacco research recruitment. Studies have found social media advertising recruitment to be more efficient for recruiting smokers when compared to specialized mailings [
Most studies examining social media advertising recruitment have used Facebook [
Participants between 14 and 28 years old who smoked at least two cigarillos per week were considered eligible. Participants also had to read and understand English and have access to an internet-connected device. Study procedures were approved by the Case Western University Institutional Review Board, and participants provided informed consent. Our study did not collect individually identifiable data from social media platforms, reducing risks related to social media data privacy and anonymity recently raised in the research community [
Advertisements featured planned images representing characteristics we intended to evaluate for advertising effectiveness (a cigar product, a smoking individual, a group of people, and gender). Advertisements were targeted to geographic areas reporting a high prevalence of cigarillo use (Cuyahoga County, OH; Baltimore, MD; Broward County, FL; Detroit, MI; DeKalb County, GA; Houston, TX; Philadelphia, PA; Washington, DC, Duval County, FL; and Fort Worth, TX) [
A budget of $240/day was set, and advertisements were scheduled to run with identical budgets so that advertisements would be delivered with similar frequency by the advertising platform algorithm. This method was essential to allow a fair comparison of advertisements as less effective advertisements may run for a brief time when using social media advertising optimization algorithms. We used the cost/click method of payment traditionally used by advertisers wishing to direct users to a website whereby costs are based on an online auction for individual clicks on the advertisement.
Facebook advertisement examples.
Individuals who completed the screener and met the eligibility criteria were invited to take the survey. Invitations were delivered via a brief email or text message, based on participant preference, and included a link to a survey administered via Qualtrics. Reminders were sent via the preferred method at 2, 5, and 8 days after initially sending the survey link. Respondents who submitted valid surveys were remunerated with a $15 gift card.
The survey included questions assessing several dimensions relevant to cigarillo consumption, including demographic information, nicotine dependence, brand preference, group smoking behavior, psychosocial constructs, and the sequence of product initiation. Several sections of the survey used branching logic such that for each tobacco product endorsed, additional questions were assessed. Therefore, the number of survey items varied based on participants’ reported product use.
Because nomenclature for tobacco products varies across audiences and geographically [
Multiple strategies were used to ensure the quality of online survey data, which can be adversely affected by respondents who falsely complete surveys for financial gain [
Completed surveys were manually assessed by study staff for validity before analysis. Validation was performed through the use of “trap” or “red herring” questions [
Data on advertising effectiveness were obtained objectively from the Facebook/Instagram and Twitter advertising analytics platforms. Data included advertisement impressions (number of individuals to whom the advertisement was delivered), cost, number of times an advertisement was clicked, and number of times an advertisement was shared on the social media platform. The advertisement that generated each survey was identified by linking the response to a unique URL. Survey data included demographics and metadata related to survey and screener completion.
Statistical analyses were conducted with SAS (Version 9.2) and SPSS (Version 25). Descriptive statistics were used to assess participant characteristics and sharing rates. Advertising effectiveness by platform and content is expressed as ratios. These included the number of screeners started per impression, the number of valid surveys per impression, and the cost per started screener and valid survey. We chose started screeners and valid surveys as the indicators of recruitment effectiveness except for the analyses of Facebook vs Instagram, where clicks were used as a metric of effectiveness. We also calculated the time in hours from when a participant started a screener to when they completed a survey. Differences between advertisement effectiveness were examined using Pearson chi-square tests. Analysis of which participant characteristics predicted recruitment advertising platform was performed using logistic regression models. Differences in survey time to completion were examined using
The combined social media advertising campaign generated 4,026,453 impressions, and 8287 started screeners. Of 6327 completed screeners, 3150 (50%) were identified as eligible and were sent a link to the survey (
Screener to survey completion flowchart.
Analysis of the predictors of social media recruitment platform is detailed in
Gender and race/ethnicity of survey completers did not predict through which social media platform a participant was recruited. The odds of being recruited by Facebook (n=310) versus Twitter (n=659) were 1.5-fold higher for those aged ≥21 years when compared to adolescents. Those recruited via Facebook were also more likely to report lower levels of education.
Demographic characteristics and tobacco use of valid survey completers.
Characteristic | Total valid survey completers (N=1089), n (%) | Valid survey completers recruited via social media (N=972), n (%) | |
|
|
|
|
|
14-18 | 19 (1.7) | 15 (1.54) |
|
19-24 | 231 (21.2) | 202 (20.78) |
|
25-28 | 839 (77.0) | 755 (77.67) |
|
|
|
|
|
Female | 509 (46.7) | 448 (46.1) |
|
Male | 568 (52.2) | 514 (52.9) |
|
Other | 12 (1.1) | 10 (1.0) |
|
|
|
|
|
Black or African American | 335 (30.8) | 309 (31.9) |
|
Hispanic | 196 (18.0) | 170 (17.5) |
|
White | 436 (40.1) | 385 (39.7) |
|
Other | 120 (11.0) | 106 (10.9) |
|
|
|
|
|
<GED or high school | 109 (10.0) | 98 (10.1) |
|
GED or high school | 235 (21.6) | 199 (20.5) |
|
Some college | 432 (39.7) | 396 (40.8) |
|
Associate | 93 (8.5) | 83 (8.6) |
|
BA+ | 219 (20.1) | 195 (20.1) |
|
|
|
|
|
Number of cigarillos smoked/day (median) | 0.93 | 0.90 |
|
Cigarette smoker | 585 (53.7) | 520 (53.5) |
|
Smoked at least one other combustible tobacco product | 866 (79.5) | 772 (79.4) |
Advertising effectiveness by platform, image type, and cost (N=4,026,453).
Advertisement type | Impressions, n (%) | Started screeners | % | Chi-square ( |
|||
All | 4,026,453 (100) | 8,287 |
|
|
|
||
|
|
|
|
|
|
||
|
2,998,715 (74) | 6,203 | .21 | 0.62 (1) | .44 | ||
|
1,027,738 (26) | 2,084 | .20 |
|
|
||
|
|
|
|
|
|
||
|
Female | 1,303,549 (32) | 2,249 | .17 | 15.55 (1) | <.001 | |
|
Male | 1,398,260 (35) | 2,142 | .15 |
|
|
|
|
|
|
|
|
|
||
|
Cigarillo | 682,994 (17) | 3,196 | .47 | 941.75 (3) | <.001 | |
|
Smoking | 1,308,675 (32) | 2,174 | .17 |
|
|
|
|
Nonsmoking | 1,393,134 (35) | 2,217 | .16 |
|
|
|
|
Group | 641,650 (16) | 700 | .11 |
|
|
Advertising effectiveness continued.
Advertisement type | Valid surveys | % | Chi-square ( |
Cost/started screener | Cost/valid survey | ||||||||
All | 972 |
|
|
|
$4.73 | $40.34 | |||||||
|
|
|
|
|
|
|
|||||||
|
661 | 0.02 | 21.45 (1) | <.001 | $4.63 | $43.41 | |||||||
|
311 | 0.03 |
|
|
$5.05 | $33.82 | |||||||
|
|
|
|
|
|
|
|||||||
|
Female | 246 | 0.02 | 0.54 (1) | .38 | $5.65 | $51.69 | ||||||
|
Male | 247 | 0.02 |
|
|
$6.17 | $53.54 | ||||||
|
|
|
|
|
|
|
|||||||
|
Cigarillo | 397 | 0.06 | 133.73 (3) | <.001 | $2.19 | $17.62 | ||||||
|
Smoking | 254 | 0.02 |
|
|
$5.99 | $51.26 | ||||||
|
Nonsmoking | 239 | 0.02 |
|
|
$5.83 | $54.06 | ||||||
|
Group | 82 | 0.01 |
|
|
$8.97 | $76.54 |
Odds of recruitment via Facebook vs Twitter (N=969).
Demographic | Coefficient | SE | ORa (95% CI) | ||||||
Male gender (vs female and other) | –0.282 | .145 | .06 | 0.755 (0.568-1.002) | |||||
Age 21-28 (vs 14-20) | 0.372 | .180 | .04 | 1.451 (1.019-2.065) | |||||
|
|
|
|
|
|||||
|
Black or African American | –0.247 | .176 | .16 | 0.781 (0.553-1.104) | ||||
|
Hispanic | 0.104 | .198 | .60 | 1.110 (0.752-1.637) | ||||
|
Other | 0.113 | .239 | .64 | 1.120 (0.701-1.788) | ||||
|
|
|
|
|
|||||
|
<GED or high school | 1.422 | .304 | <.001 | 4.147 (2.286-7.521) | ||||
|
GED or high school | 1.447 | .250 | <.001 | 4.250 (2.603-6.939) | ||||
|
Some college | 0.983 | .228 | <.001 | 2.672 (1.710-4.173) | ||||
|
Associate’s degree | 1.019 | .306 | <.001 | 2.770 (1.519-5.050) |
aOR: odds ratio.
This study was successful in recruiting a large sample of youth and young adult cigarillo users, supporting the use of social media-based advertising to reach this group, which traditionally presents a challenge to survey and study recruitment efforts. The recruited sample was diverse and representative of individuals most likely to use cigarillos, with over a third of our sample consisting of African Americans, a demographic group with the highest reported cigarillo use among adults [
Advertisements on Facebook were more effective than Twitter in producing valid surveys. Facebook was also a more cost-effective recruitment method. Differences observed in the cost per approved survey (approximately $10 US) could represent a significant increase in the overall costs of recruitment in extensive studies. A limited number of previous studies have compared Facebook and Twitter research recruitment. A feasibility study examining the use of QR codes, Facebook, and Twitter for recruiting adolescents to take a health-related survey found Twitter to be marginally more cost-effective than Facebook [
Our cost per valid survey of $40 US falls within a broad range of previous findings related to the cost-effectiveness of social media recruitment of tobacco users. Ramo et al reported substantially lower Facebook costs of $8.80 US per eligible, consented participant in a tobacco cessation intervention and $4.28 per completed survey in a separate study, both targeting young adults [
Our results also demonstrate that differences in effectiveness are a function of the metric used. Twitter was as effective in generating started screeners as Facebook but was less effective in generating approved surveys. Investigators should use the measure of effectiveness most closely related to their recruitment goal as opposed to more distal measures such as advertising clicks that are more accessible via social media advertising platforms. Participants recruited via Facebook also completed the process of producing a valid survey more rapidly than those recruited via Twitter. For large studies or those on a short timeline, this measure of performance should be considered to ensure that the study goals are met. We observed minimal naturally occurring sharing of advertisements across platforms when compared to the reach of paid advertising. The viral nature of social media platforms, however, lends itself to this mode of recruitment, and researchers are beginning to use this strategy for a variety of recruitment and communications tasks [
Regardless of the platform, the inclusion of a cigarillo image was consistently the most effective advertisement design. A similar result was found in a comparison of Facebook advertisement images used to recruit young smokers, where the advertisement with an image of a cigarillo had the lowest cost per unique click of 4 images used [
This study has several strengths. We had a large sample of advertising impressions on which to base our analysis. We created individual URLs for each advertisement on each platform linked to individual participants’ screeners and surveys, allowing us to examine more relevant forms of effectiveness and assess demographic factors associated with social media platform advertising effectiveness. Limitations include our inability to examine Facebook and Instagram separately at the same level of effectiveness as our other analyses. The demographic profile of these social media platforms differs on several key demographic factors such as age and race, which may create differences in their effectiveness as suggested by our results comparing clicks per impression. In addition, over the past five years, Instagram use has consistently increased while Facebook has leveled off [
Social media platforms should be considered for tobacco-related research, especially in studies attempting to recruit hard-to-reach populations, such as youth, young adults, and minorities. This method is cost-effective when compared to other modalities and provides a convenient means of digital recruitment on a nationwide basis. Although in the current study, Facebook was a more cost-effective platform, given the fact that we observed some differences in the likelihood of recruitment platform between age and education levels and our inability to determine how many individuals in the sample use both, it may be that using multiple platforms remains the optimal recruitment strategy. Using multiple social media platforms may be less cost-effective, but by increasing the potential audience size, researchers may improve the ability to reach subgroups and recruit rapidly, if required.
This project was funded by a grant from NIH, the National Cancer Institute, and the Center for Tobacco Prevention, R01CA109130.
None declared.