This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Web-based recruitment for research studies is becoming increasingly popular and necessary. When compared with the traditional methods of recruitment, these methods may enable researchers to reach more diverse participants in less time. Social media use is highly prevalent among adolescents, and the unique context of social media may be particularly important for the recruitment of sexual minority young people who would not be captured by traditional methods.
This paper described the details of a national web-based study recruitment approach aimed at sexual minority adolescents across the United States, focusing on important details of this relatively novel approach, including cost, time efficiency, and retention outcomes.
This study recruited sexual minority adolescents aged 14-17 years living in the United States through targeted advertisements on Facebook, Instagram, and YouTube and through respondent-driven sampling (RDS). Potential participants completed eligibility screening surveys and were automatically directed to a baseline survey if they were eligible. After baseline survey completion, additional data checks were implemented, and the remaining participants were contacted for recruitment into a longitudinal study (surveys every 6 months for 3 years).
Recruitment lasted 44 weeks, and 9843 participants accessed the initial screening survey, with 2732 (27.76%) meeting the eligibility criteria and completing the baseline survey. Of those, 2558 (93.63%) were determined to have provided nonfraudulent, usable study data and 1076 (39.39%) subsequently enrolled in the longitudinal study. Of the baseline sample, 79.05% (2022/2558) was recruited through Facebook and Instagram, 3.05% (78/2558) through YouTube, and 17.9% (458/2558) through RDS. The average cost of recruiting a participant into the study was US $12.98, but the recruitment cost varied by method or platform, with a realized cost of US $13 per participant on Facebook and Instagram, US $24 on YouTube, and US $10 through RDS. Participant differences (sex assigned at birth, race and ethnicity, sexual orientation, region, and urbanicity) were identified between platforms and methods both in terms of overall number of participants and cost per participant. Facebook and Instagram were the most time efficient (approximately 15 days to recruit 100 participants), whereas RDS was the least time efficient (approximately 70 days to recruit 100 participants). Participants recruited through YouTube were the most likely to be longitudinally retained, followed by Facebook and Instagram, and then RDS.
Large differences exist in study recruitment cost and efficiency when using social media and RDS. Demographic, region, and urbanicity differences in recruitment methods highlight the need for attention to demographic diversity when planning and implementing recruitment across platforms. Finally, it is more cost-effective to retain than recruit samples, and this study provided evidence that with thorough screening and data quality practices, social media recruitment can result in diverse, highly involved study populations.
The use of web-based recruitment (including social media, websites, and other internet-based platforms) for research studies has become an increasingly popular tool in recent years [
Criticisms for web-based recruitment are not without merit. For instance, although these methods may recruit large samples in relatively short periods, they may be more or less effective with different populations. For example, Carter-Harris et al [
Despite some challenges, these recruitment methods may be particularly useful in certain populations, which are more difficult to reach through probability sampling approaches. In particular, an overwhelming majority of young people use social media sites. For example, Barry et al [
A unique feature of web-based recruitment, particularly through social media platforms, is the ability to access hard-to-reach samples. In particular, recruitment and retention of sexual minority adolescents may be especially challenging because of stigma or discrimination based on their sexual orientation [
Another recruitment method to access hard-to-reach populations is respondent-driven sampling (RDS; [
Given the growing evidence supporting the utility of the web-based recruitment of research participants, known difficulties in the existing methods for recruiting sexual minority adolescents given their sometimes-hidden status, and the fact that most adolescents use social media and other web-based platforms, this paper outlines the recruitment process and lessons learned in a large-scale, longitudinal study of sexual minority adolescents. This paper aimed to describe in detail our web-based recruitment approach and present quantitative data on cost and retention outcomes, including the cost of all types of recruitment, whether direct web-based (via targeted advertising) or RDS recruitment is more cost-efficient, and which platform (Facebook and Instagram vs YouTube) is more cost-efficient. We also present our findings on the “true” cost of web-based and RDS recruitment and retention in a national sample of hard-to-reach adolescents over 2 years, which method was most successful at retention, and whether demographic characteristics affected any of these processes.
Ethics review and approval was obtained by the The Washington University in St Louis Institutional Review Board (IRB ID: 202212035).
This study recruited adolescents aged 14 to 17 years in the United States who identified as nonheterosexual and cisgender male or female through targeted advertisements on Facebook, Instagram, and YouTube channels and through RDS. Recruitment for baseline participation lasted 44 weeks between May 2018 and April 2019. Facebook and Instagram advertisements went live on May 15, 2018, and YouTube advertisements went live on January 9, 2019. RDS was available to all the participants and ran for the entire duration of 44 weeks that the survey was open.
Facebook’s Ads Manager (which delivers advertisements for both Facebook and Instagram) allows targeting by users’ demographics (eg, age, gender, and location) and interests (eg, activities or sports [
After starting Facebook and Instagram recruitment, our youth advisory group indicated that their peers used YouTube as another common source of information. We started expanding our recruitment to YouTube by using keywords such as “LGBTQ,” “gay,” and “coming out” to identify YouTube channels relevant to the target population. Channels with the highest subscriber counts were further evaluated to identify large visibility, reach, and engagement (operationalized as the number of video views for each channel’s top 3 most-viewed videos). Of the 47 possible YouTube channels identified as having high visibility and engagement among LGBTQ adolescents, 23 (49%) were available for advertising and were confirmed as relevant to sexual minority adolescents per discussions with youth advisors. The Google advertising system was used to place these advertisements on the YouTube channel pages identified.
Similar advertisements were used on both Facebook and Instagram and YouTube; the advertisement text asked those who saw the advertisement to “Share Your Voice!” Information about the study was also included: “Be on the forefront of LGBTQ research and earn up to $195! We’re doing a research study to learn about the experience of growing up as a lesbian, gay, bisexual, or pansexual young person. If you’re eligible, you’ll receive a $15 gift card for taking the first survey, and some people will be invited to be part of future studies with additional gift cards. Click here to start.” Each advertisement was accompanied by 2 images. The first image in all the advertisements was a city skyline with a rainbow flag and the words “Share Your Voice,” and the second image was tailored to boys or girls, depicting either a young man or a young woman “shouting” the “Share Your Voice” tagline.
A participant who enrolled in the study had the opportunity to recruit up to 3 people like them and received a monetary incentive of a US $10 gift card for each successful recruit. Those who confirmed their interest in recruiting others received an email that contained 3 unique links, each to be sent to only one referral. A prompt offered language that the participant could use to send the links to their friends.
Potential participants who clicked an advertisement from one of the social media platforms or were recruited through RDS were routed to a Qualtrics (Qualtrics International Inc) survey page. Screener questions were asked to determine eligibility. Eligible participants were aged between 14 and 17 years, identified as the same gender assigned at birth (ie, cisgender), and were currently living in the United States. Adolescents also had to identify as nonheterosexual based in an item assessing sexual orientation (ie, identify as “Mostly heterosexual [straight], but somewhat attracted to people of your own gender”; “Bisexual or pansexual”; “Mostly homosexual [gay/lesbian] but somewhat attracted to people of the opposite gender”; “100% homosexual [gay/lesbian]”; or “Unsure”). These eligibility requirements were set following similar guidelines used in the development and validation of the Sexual Minority Adolescent Stress Inventory (SMASI) [
Rigorous steps were taken to identify any fraud (eg, duplicate participants and participants who failed the screener and reaccessed the survey with different answers) and poor-quality data (low validation or attention scores and low survey duration). Of the 9843 participants who initially accessed the survey, 2732 (27.76%) met the eligibility criteria for inclusion in the study. Steps were taken to ensure the best-quality data. Participants were denied access to the full survey if they were ineligible based on screener questions, closed the survey before the end of the screener, or did not assent to participate. Postsurvey data-cleaning efforts resulted in the removal of participants who closed the survey before its completion, made multiple attempts to take the survey, were not in the payment survey, or accessed the survey through a fraudulent channel. Further cleaning was then conducted to exclude those with fraudulent or extremely poor-quality data, such as unrealistically short survey duration, low validation or attention control scores (eg, “Please select the response option ‘None of the above’” answered incorrectly), successful repeat survey completions by the same individual, or missing data on key outcome measures [
Participants who were eligible for the baseline study and indicated an interest in participating in the longitudinal study were enrolled for follow-up surveys (1076/2732, 39.39%). Participants were contacted every 6 months based on the date of the baseline survey. At each survey, participants assented or consented to the survey and received gift cards that increased by US $5 at each wave. To keep participants engaged during the study and ensure that contact information stayed up-to-date, check-in surveys were sent every month. A single question determined whether their contact information had changed, and participants had the opportunity to update it. Respondents who participated in the check-ins were entered into a raffle to win US $100 each time they completed the monthly survey.
Of the 2558 participants who were successfully enrolled and retained in the initial study, 2022 (79.05%) were recruited through Facebook and Instagram, 78 (3.05%) were recruited through YouTube, and 458 (17.9%) were recruited through RDS.
Advertisements on Facebook and Instagram remained active during the 44 weeks of the study and cost US $26,571.62. The survey was advertised 3,035,126 times and reached the profiles of 617,927 unique people, of whom 18,469 (2.99%) clicked the advertisement to access the survey. This resulted in a nominal cost of US $1.44 per click. Of these 18,469 individuals, 7773 (42.09%) individuals accessed the survey, which began with screening questions after clicking the advertisement. Steps were taken to ensure the best-quality data. Participants were denied access to the full survey if they were ineligible based on screener questions, closed the survey before the end of the screener, or did not assent to participate. Data from 121 Facebook and Instagram participants were excluded for these reasons, resulting in 2022 Facebook and Instagram participants—representing 10.9% of all advertisement clicks (n=18,469) on these platforms—who successfully completed the study, bringing the realized (adjusted) cost of Facebook and Instagram advertising to US $13.14 per participant.
In comparison, advertisements on YouTube were only active for 3 weeks of the study for a total cost of US $1892.89. Of 1621 clicks, most came from a single channel (n=960, 59.22%); they yielded a nominal cost of US $1.16 per person. Of the 1621 clicks, 538 (33.19%) individuals accessed the survey. However, after data screening and cleaning efforts similar to those previously described, only 78 (4.81%) eligible participants completed the survey after recruitment through YouTube advertisements, resulting in a realized cost of US $24.27 per participant. James Charles, a beauty vlogger and makeup artist, brought in the most eligible participants for a single channel (29/78, 37%).
Finally, participants who shared RDS referral links brought in 473 additional survey responses and received US $10 for their eligible referrals. Unlike direct advertising, in which the cost of the advertisement is incurred before the potential participant accesses the survey, RDS referrals only triggered referral payments after the referred participant successfully screened as eligible and completed the survey. Thus, the adjusted cost of the 458 (96.83%) out of 473 participants whose data were retained for the study was US $10.33. Overall, the average adjusted cost of recruiting a single participant into the study was US $12.98, with RDS being the most cost-efficient method.
In addition to the costs associated with attracting potential participants to the study, participants received US $15 for completing the baseline survey. The total payout was US $40,980 to 2732 participants who were initially screened as eligible for study participation based on the study inclusion criteria. After removing data from participants whose efforts were determined to be fraudulent or of extremely poor quality as described previously, 2558 participants were retained in the baseline sample, resulting in a realized cost per retained baseline survey of US $16.02. Combined with the associated recruitment costs, this resulted in a total realized cost of US $29.00 per baseline study participant.
Costs associated with recruitment, participation, and retention by initial recruitment method.
Phase | Information | Total cost (US $) | Cost description | Unique participants paid, n | Participants retained, n (%)a | Nominal cost per participant (US $) | Realized (adjusted) cost per participant (US $) | Cost difference (US $) |
Advertising recruitment | Facebook and Instagram | 26,571.62 | 18,469 clicks | 2143 | 2022 (10.9b) | 1.44 | 13.14 | 11.69 |
Advertising recruitment | YouTube | 1892.89 | 1621 clicks | 81 | 78 (4.8c) | 1.16 | 24.27 | 23.11 |
RDSd recruitment | Referrals | 4730 | 473 referrers at US $10 per referral | 508 | 458 (90.2e) | 10.00 | 10.33 | 0.33 |
Study enrollment | Baseline | 40,980 | US $15 baseline survey incentive | 2732 | 2558 (93.6) | 15.00 | 16.02 | 1.02 |
Retention | 6 months | 19,940 | US $20 6-month survey incentive | 997 | 969 (97.2) | 20.00 | 20.58 | 0.56 |
Retention | 12 months | 24,025 | US $25 12-month survey incentive | 961 | 945 (98.3) | 25.00 | 25.42 | 0.40 |
Retention | 18 months | 27,570 | US $30 18-month survey incentive | 919 | 912 (99.2) | 30.00 | 30.23 | 0.20 |
Retention | 24 months | 31,430 | US $35 24-month survey incentive | 898 | 894 (99.6) | 35.00 | 35.16 | 0.12 |
Retention | Raffles | 3100 | US $100 monthly raffle prize | 31 | 31 (100) | 100.00 | 100.00 | 0.00 |
aThe denominator of each percentage is the number of unique participants paid in the same row, except for those that indicate otherwise.
b10.9% of attempts out of 18,469 clicks.
c4.8% of attempts out of 1621 clicks.
dRDS: respondent-driven sampling.
e90.2% of referrals out of 473 referrers at US $10 per referral.
The large cost discrepancies that existed across these methods may also shed light on which avenues may be more successful at accessing young people with specific demographic characteristics. Because Facebook and Instagram drew most participants who were eligible for the study, few discrepancies were observed in that group compared with the overall sample. However, both YouTube and RDS provided insight into the categories of people who may be more inclined to access the survey through these methods.
Demographics of participating youths (N=2558).
Demographics | Total, n (%) | FBa and IGb (n=2022), n (%) | YouTube (n=78), n (%) | RDSc (n=458), n (%) | Chi-square ( |
||||||||||
|
44.2 (1) | <.001 | |||||||||||||
|
Male | 912 (35.65) | 779 (38.53) | 7 (8.97) | 126 (27.51) |
|
|
||||||||
|
Female | 1646 (64.35) | 1243 (61.47) | 71 (91.03) | 332 (72.49) |
|
|
||||||||
|
28.4 (3) | <.001 | |||||||||||||
|
14 | 260 (10.16) | 198 (9.79) | 16 (20.51) | 46 (10.04) |
|
|
||||||||
|
15 | 568 (22.2) | 425 (21.02) | 26 (33.33) | 117 (25.55) |
|
|
||||||||
|
16 | 888 (34.71) | 700 (34.62) | 19 (24.36) | 169 (36.9) |
|
|
||||||||
|
17 | 842 (32.92) | 699 (34.57) | 17 (21.79) | 126 (27.51) |
|
|
||||||||
|
43.1 (5) | <.001 | |||||||||||||
|
Native American, American Indian, or Alaska Native | 59 (2.31) | 50 (2.47) | 0 (0) | 9 (1.97) |
|
|
||||||||
|
Asian or Pacific Islander | 164 (6.41) | 120 (5.93) | 7 (8.97) | 37 (8.08) |
|
|
||||||||
|
Black or African American | 199 (7.78) | 151 (7.47) | 1 (1.28) | 47 (10.26) |
|
|
||||||||
|
White | 1548 (60.52) | 1269 (62.76) | 51 (65.38) | 228 (49.78) |
|
|
||||||||
|
Latino or Hispanic | 370 (14.46) | 261 (12.91) | 11 (14.1) | 98 (21.4) |
|
|
||||||||
|
Multiracial | 217 (8.48) | 170 (8.41) | 8 (10.26) | 39 (8.52) |
|
|
||||||||
|
39.3 (4) | <.001 | |||||||||||||
|
Mostly heterosexual | 137 (5.36) | 92 (4.55) | 2 (2.56) | 43 (9.39) |
|
|
||||||||
|
Bisexual or pansexual | 1035 (40.46) | 785 (38.82) | 42 (53.85) | 208 (45.41) |
|
|
||||||||
|
Mostly homosexual | 491 (19.19) | 400 (19.78) | 17 (21.79) | 74 (16.16) |
|
|
||||||||
|
100% homosexual | 850 (33.23) | 710 (35.11) | 16 (20.51) | 124 (27.07) |
|
|
||||||||
|
Unsure | 45 (1.76) | 35 (1.73) | 1 (1.28) | 9 (1.97) |
|
|
||||||||
|
44.9 (4) | <.001 | |||||||||||||
|
Gay | 618 (24.16) | 541 (26.76) | 6 (7.69) | 71 (15.5) |
|
|
||||||||
|
Lesbian | 440 (17.2) | 329 (16.27) | 19 (24.36) | 92 (20.09) |
|
|
||||||||
|
Bisexual or pansexual | 1166 (45.58) | 887 (43.87) | 42 (53.85) | 237 (51.75) |
|
|
||||||||
|
Queer | 85 (3.32) | 72 (3.56) | 4 (5.13) | 9 (1.97) |
|
|
||||||||
|
Other | 249 (9.73) | 193 (9.54) | 7 (8.97) | 49 (10.7) |
|
|
||||||||
|
34.9 (4) | <.001 | |||||||||||||
|
West | 630 (24.63) | 465 (23) | 32 (41.03) | 133 (29.04) |
|
|
||||||||
|
Southwest | 336 (13.14) | 281 (13.9) | 8 (10.26) | 47 (10.26) |
|
|
||||||||
|
Midwest | 448 (17.51) | 380 (18.79) | 14 (17.95) | 54 (11.79) |
|
|
||||||||
|
Southeast | 592 (23.14) | 465 (23) | 11 (14.1) | 116 (25.33) |
|
|
||||||||
|
Northeast | 552 (21.58) | 431 (21.32) | 13 (16.67) | 108 (23.58) |
|
|
||||||||
|
33.9 (1) | <.001 | |||||||||||||
|
Rural | 504 (19.7) | 446 (22.06) | 12 (15.38) | 46 (10.04) |
|
|
||||||||
|
Urban | 2054 (80.3) | 1576 (77.94) | 66 (84.62) | 412 (89.96) |
|
|
aFB: Facebook.
bIG: Instagram.
cRDS: respondent-driven sampling.
Most participants were female (1646/2558, 64.35%), with similar distributions for both Facebook and Instagram (1243/2022, 61.47%) and RDS (332/458, 72.5%). However, 91% (71/78) of the participants recruited via YouTube were female (
Sexual identity also showed differences in recruitment methods (
Differences in region (
The Facebook Ads Manager provided more detailed information on the breakdown of cost per targeted group. Advertisements targeted 3 characteristics for equal distribution, gender, region, and urbanicity, resulting in the 20 unique groups described previously. Facebook allowed bulk uploading of zip codes for target locations (ie, region and urbanicity), but only up to 2500 zip codes were allowed per advertisement. Multiple advertisements were created for groups with > 2500 zip codes (eg, urban girls in the West), resulting in 44 targeted advertisements. All 44 advertisements could be advertised on a single day, resulting in 1286 “ad days.”
Facebook and Instagram advertising cost by target groupa.
|
Total cost (US $) | Number of days open | Number of reaches | Cost per reach (US $) | Cost for recruitment (US $) | ||||||
|
|||||||||||
|
Male | 15,043.61 | 724 | 237,384 | 0.06 | 16.48 | |||||
|
Female | 11,528.01 | 562 | 380,543 | 0.03 | 7.00 | |||||
|
|||||||||||
|
West | 4663.43 | 189 | 101,435 | 0.05 | 7.39 | |||||
|
Southwest | 5468.43 | 123 | 139,590 | 0.04 | 16.28 | |||||
|
Midwest | 6062.16 | 238 | 139,880 | 0.04 | 13.53 | |||||
|
Southeast | 5643.79 | 396 | 126,643 | 0.04 | 9.53 | |||||
|
Northeast | 4733.81 | 340 | 110,379 | 0.04 | 8.57 | |||||
|
|||||||||||
|
Rural | 14,588.29 | 625 | 200,385 | 0.07 | 28.89 | |||||
|
Urban | 11,983.33 | 661 | 417,542 | 0.03 | 5.83 |
aFacebook Ads Manager allows targeting by zip code and gender but not by other demographic characteristics.
Advertisements targeting girls cost US $11,528.01 and spanned 562 ad days, resulting in a cost of US $20.51 for each day the advertisement was visible. These advertisements were seen by 380,543 unique girls, costing US $0.03 per view. With the successful recruitment of 1646 female participants into the study, this group cost an average of US $7.00 (SD $2.33) per participant in advertising to recruit. Advertisements targeting boys cost more, at US $15,043.61, but resulted in a daily cost of US $20.78, which is similar to that of the advertisements targeting girls because the advertisements targeting boys were open for a longer period (724 days). These advertisements reached 237,384 unique male participants and resulted in a cost of US $0.06 per view, which is twice that of the advertisements for female participants. Similarly, with 913 boys enrolled in the study, successfully recruiting a male participant through an advertisement on Facebook and Instagram cost US $16.48, more than twice as much as the cost for successfully recruiting a female participant on these platforms. The recruitment of male participants through Facebook and Instagram both took longer and cost more than that of their female counterparts (
The cost of advertisements significantly differed across regions (all comparisons:
The overall cost by recruitment methods was calculated as the realized cost of advertising or referral plus the adjusted per-participant cost of the baseline survey. Advertisements on Facebook and Instagram resulted in a total cost of US $29.15 per participant, those on YouTube resulted in a total cost of US $40.28 per participant, and RDS resulted in a total cost of US $26.31 per participant. RDS was a significantly more cost-efficient recruitment method compared with Facebook and Instagram (
To better understand and compare the cost, time, and participant exclusion rates by recruitment method, the values were standardized to 100 participants (
Comparison of expected cost, time, and exclusion rate per 100 participants by recruitment method.
Recruitment method | Cost (US $) | Time to recruit (days) | Excluded participants (per 100) |
Facebook and Instagram | 1314 | 15.22 | 5.6 |
YouTube | 2427 | 26.95 | 3.7 |
RDSa | 1031 | 69.93 | 9.6 |
aRDS: respondent-driven sampling.
Of the 2558 participants with retained baseline survey data, 2073 (81.04%) indicated an interest in being contacted for additional research opportunities and were offered enrollment in the longitudinal portion of the study. Among these 2073 participants, 934 (45.06%) were lost to contact, 49 (2.36%) declined participation, and 14 (0.68%) were excluded from follow-up for other reasons (eg, not being able to remember the study password that ensured that the research staff were contacting the right participant). Thus, 51.91% (1076/2073) of participants were successfully recruited into the longitudinal portion of the study. A total of 5 waves of data and 4 follow-up surveys occurred during the first 2 years of data collection, with follow-ups gathered at 6 months, 12 months, 18 months, and 24 months.
Participation completion.
To keep longitudinally enrolled participants engaged during the study, monthly requests to provide updated contact information were sent to all participants with an associated raffle drawing. The cost of each monthly raffle is included in
Of the 1076 participants who were longitudinally enrolled in the study, 874 (81.23%) came in through Facebook and Instagram, 45 (4.18%) came in through YouTube, and 157 (14.59%) came in through RDS. These findings are similar to those related to the recruitment of the 2558 participants enrolled in the baseline survey. However, those who were longitudinally enrolled and those who were not longitudinally enrolled had significant differences. Those recruited through YouTube were more likely to be retained longitudinally than those in the Facebook and Instagram group (odds ratio [OR] 1.79, 95% CI 1.45-2.21;
Relatively minor differences existed among the longitudinally enrolled participants across demographics, demonstrating success in keeping a diverse group of participants over time. By 24 months, 86.8% (624/719) of the female participants remained in the study compared with 76.2% (272/357) of the male participants (
A national study with 2 years of follow-up data was used to examine the true cost of recruitment and retention of sexual minority adolescents. Participants were recruited through Facebook and Instagram advertisements, YouTube advertisements, and RDS referrals not only to access this hard-to-reach population but also to understand which method may be the most cost-efficient, time efficient, and successful in recruiting participants for the longitudinal portion of the study. Although the nominal cost of advertising based on the number of individuals approached to participate in the study was only US $2.97, after accounting for ineligible youths who viewed and interacted with advertisements, fraudulent attempts at study participation, and the exclusion of low-quality data, the realized cost of attracting an eligible person to the study averaged to US $12.97. It should be noted that a challenge in comparing our results to other published studies (including systematic reviews; eg, Whitaker et al [
Large expenses and discrepancies between nominal and realized costs existed in the recruitment process; RDS was the most cost-efficient method, and RDS and Facebook and Instagram advertising were both more cost-efficient than YouTube advertising. However, Facebook and Instagram were much more time efficient than RDS, bringing in 79.05% (2022/2558) of the study participants compared with 17.9% (458/2558) via RDS during the 44 weeks. It is perhaps not surprising that RDS was the most effective approach, as it relies upon peer referral and is, therefore, a more trusted source. Facebook and Instagram (largely driven by Instagram) likely come next because (1) youths in our target demographic are heavily represented on Instagram and (2) the algorithms used by Facebook and Instagram to target potential participants are highly effective in ensuring that the appropriate audience has access to study recruitment materials. As YouTube advertising focuses on channels, we relied largely upon input from youths in the target demographic based on which channels they most frequently watched. In the absence of a well-constructed algorithm of users (which YouTube has less of, given that no log-in is required to view most videos), this meant that our advertisement was not necessarily being seen by only the target audience and was, therefore, resulting in more clicks that had, however, far less specificity.
Demographics played a large role in both the type and cost of recruitment. Each characteristic grouping contributed to statistically significant differences in type of recruitment: gender, age, sexual attraction, sexual orientation, race and ethnicity, region, and urbanicity. Furthermore, differences in the cost of recruitment based on gender, region, and urbanicity were observed for those recruited through Facebook and Instagram. Our findings are similar to those of the systematic review by Whitaker et al [
Nominal (US $15) and real (US $16.01) costs for the completion of the baseline survey showed small differences attributable to data exclusion after payment, and each of the 4 successive waves lessened the gap between these 2 costs. Although no cost differences existed in the longitudinal portion of the sample, as everyone received the same amount at each completed wave, differences were observed in the initial recruitment method in longitudinal enrollment and retention. Those recruited through RDS were surprisingly less likely to enroll longitudinally, whereas those recruited through YouTube were the most likely to enroll longitudinally. Although it is difficult for us to understand the exact reason for this change, future research should ask participants how the platform may influence their behaviors in enrolling and maintaining participation in research studies. Demographic differences were not found to vary in meaningful ways, suggesting that these strategies may be used to multiply the number of marginalized youths participating in research studies (eg, racial and ethnic minority and rural youths).
Importantly, despite the large differences between nominal and actual costs of recruitment, data-screening practices are necessary to ensure excellent-quality data with large internet samples [
It should be noted that the advertising recruitment methods outlined in this study reflect the capacities of advertising platforms available for Facebook and Instagram and YouTube at the time of our recruitment activities (May 2018 to April 2019). Given the growing concerns about privacy and disinformation on social media, changes over time in these advertising platforms are inevitable, particularly regarding the ability to focus on vulnerable or marginalized populations. Relevant to our findings in this paper, for example, in summer 2021, Facebook’s advertising platform discontinued the use of interest group targeting for anyone younger than 18 years [
Our study is not without limitations. Because the primary goal of this project was not to understand engagement in various recruitment media, we do not have data describing the impact of web-based recruitment on continued study engagement of adolescents compared with other methods (eg, venue-based recruitment). It is possible that some adolescents who declined longitudinal participation may have done so because of the impersonal nature of the web-based recruitment approach. Furthermore, the use of social media apps by adolescents changes regularly. Although many youths in our study indicated that they still use Facebook or Instagram, the landscape of social media is constantly shifting, and researchers will need to be mindful of this or face mounting costs as users leave platforms that have fallen out of favor.
Despite these limitations, key advantages of recruiting a sample using this approach warrant its ongoing use. First, we could target, screen, and carefully recruit a national sample within a relatively short period. Second, given the demographic differences in the cost to recruit people (eg, by age, gender, urbanicity, and sex assigned at birth), we rebudgeted to ensure successful recruitment in real time, adding and removing advertisements from the platforms in moments. This level of control proved useful, particularly because we recognized a need to expand our rural sample to address the sexual minority adolescent experience, as it applies to a more unique group that may not have the resources of those in urban settings. Finally, although the initial cost to recruit a national sample of diverse adolescents was more expensive than anticipated, the cost to retain them was similar to the nominal cost. Given the concerns regarding the ability to retain adolescents in studies over time (in general), particularly those with no direct in-person contact to build rapport, this was a welcome finding that we believe can support future studies with this and other vulnerable populations.
lesbian, gay, bisexual, transgender, and queer
odds ratio
respondent-driven sampling
Sexual Minority Adolescent Stress Inventory
None declared.