A total of 5818 records were identified during the initial searches. Duplicates were removed (n=1239) and 4579 records were screened against the exclusion criteria (Figure 1). Additionally, 123 full papers were assessed for quality using the study design specific CASP checklist revealing 35 papers (scoring 7-9/9) to be included in the review (see Multimedia Appendix 1). Quantitative and qualitative data was tabulated (Tables 2 and 3), allowing comparison of cost and demographic recruited.

Most studies were conducted in the United States (n=22) with some in Australia (n=9) and Canada (n=2) and one in Japan and Germany, respectively. Some studies also tested interventions (n=10); three recruited for smoking cessation [6,9,10], two for human papillomavirus (HPV) vaccination [11,12], two for healthier lifestyle intervention [13,14]; and one each for perinatal studies [15], human immunodeficiency virus (HIV) prevention via soap opera viewing [3], and depression intervention [16]. Ten papers recruited those aged 18 years and over, 7 the age group of 18-25 years, and 16 recruited different ages (See Tables 3 and 4 for more demographic information).

Other than basic demographic information including age and sex, most papers recruited participants with specific characteristics (n=18), including parents of children [17], nulliparous women at the beginning of their pregnancy [18], Brazilian and Portuguese speakers [19], emergency nurses [20], those with post-traumatic stress disorder (PTSD) [21], those with Klinefilter syndrome [22], those in sexual relationships [23], those living in a metropolitan area [24], US veterans [25], parents of children with Fontan-associated protein losing enteropathy [26], and those of positive HIV status [27]. Two papers [16,28] had no targeting features except being over 18 years old.

There were several pieces of data that outlay three SDs and so were removed from statistical analysis, namely, a recruitment length of 36 months [31], an impression count of 127 million [43], a cost per click of US $1.74 [38], a conversion rate of 48% [24], and a cost per participant of US $110.00 [11].

Table 5 shows median data: 264 recruited participants, a 3-month recruitment period, 3.3 million impressions, cost per click of US $0.51, conversion rate of 4% (range 0.06-29.50), eligibility of 61% (range 17-100), and cost per participant of US $14.41.

Most studies (n=24) compared their recruited participants with either a control group recruited by traditional methods or to national data. This showed the recruited participants to be relatively representative except for some minor differences:

Cost per click only varied slightly across studies, especially when targeting similar groups. The median cost per click from this review was US $0.51 compared with US $0.27—the mean cost per click on Facebook as a whole [44]. This shows people are less likely to interact with a health recruitment ad. The conversion rate of 4% can also be compared with the mean value of 2.4% across all Web-based advertising [45]. This suggests that although people are less interested in health research ads overall, those who do interact with them are more likely to convert. This increase in conversion rate however, doesn’t appear large enough to counteract the increased cost per click with health recruitment still costing more than general advertising.

The cost per participant on Facebook was shown to be less than traditional methods. Our median value of US $14.41 compares favorably with rates suggested by Tate, D (2014) of US $1094.27 per participant for television recruitment, US $811.99 for printed media, US $635.92 for radio, and US $37.77 for email when recruiting for a survey on English language competency [5]. Carlini BH (2015) had similar findings with a mean cost per participant of US $16.22 via Google ads, and between US $13.12 and US $250.00 for other traditional methods when recruiting young adults for weight gain analysis [19].

The cost per participant values contained a major outlier; Raviotta JM (2016) reported a cost of US $110 per recruited participant [11]. The cost per click of the study (US $1.24) fell slightly outside one SD of the median and did not explain the increased cost per participant. On closer inspection, the reason for the expense became clear, “the difference in time and effort required to complete a 7-13 month study with two blood draws and three vaccine injections vs. a 30 minute survey...explains the increased cost” [11].

Three articles simultaneously used other social media sites to recruit participants, namely Twitter and MySpace. Bull S (2013) used MySpace but found it unsuccessful in recruiting any participants [31]. This is unsurprising considering the massive drop in MySpace primary users from 2008 to 2011 [46]. Harris, M. L (2015) implemented recruitment via Facebook and Twitter but changed to use Facebook alone due to its increased success [34]. Yuan P (2014) also used Twitter alongside Facebook. The study received 10,006 Facebook ad clicks and 161 Twitter interactions. It was found that the number of Facebook ad clicks was moderately correlated to the number recruited (r=.52 P<.001) but that there was little correlation between Twitter interactions and links clicked (r=.17, P=.06; r=.18, P=.06, respectively) [27]. These findings suggest Facebook is a superior recruitment tool when compared with Twitter and MySpace, although there is limited analysis across the three papers. This was an interesting albeit unintentional finding, but more research should be carried out in this area before making conclusions.

Sociodemographic characteristics of the recruited participants were compared either with traditionally recruited participants or to available national statistics. Alcohol consumption; marijuana, ecstasy, and cocaine use [30]; STI rates; and sexual relationship history [35] were found to be representative of the total population. Those who use recreational drugs and have at risk sexual behavior tend to be found in hard to reach populations. The fact these studies mirror national statistics highlights the power of social media to target specific populations. Traditional methods tend to under represent these groups [47], meaning Facebook recruitment potentially yields more significant results.

Another point highlighting the success of Facebook recruitment would be the differing BMI results from Miyagi E (2014) and Fenner Y (2012), with the latter Australian paper showing a considerably higher average than the Japanese study. This simply shows the different obesity rates of the two countries. Australia reports 64% of its population to have a BMI above 25kg/m²compared with 24% in Japan [48]. This also seems true for the differences in reported rates of HPV vaccination between Nelson EJ (2014) and Remschmidt C (2014); 39.7% of adolescent females in America [49] are vaccinated compared with 49% of Germans [50].

Other demographic data was found to be representative of the target population and comparable with traditional recruitment with only a few exceptions:

There was an over representation of white ethnicity. Facebook claims to be diverse [51], but papers suggest either these claims are not true, targeted marketing misses certain ethnicities, or that different groups have different response rates. This over representation is also shown in a review of traditional methods by Yancey AK (2005), suggesting the problem is not limited to social media recruitment [52].

Four papers also showed over representation of females. This may be due to the fact that a higher percentage of women use Facebook [1] or because fewer men respond to recruitment in general [53]. Brown WJ (1998) found similar results using traditional methods, again suggesting the problem is not limited to Facebook [54].

Education and income are often confounding factors, and it is perhaps unsurprising to find over representation in both these areas. This is comparable again with traditional recruitment methods, with Gorelick PB (1998) finding that more years in education increased the likelihood of entering and completing a clinical trial with those of lower levels “not wanting to be guinea pigs” [55].

Strengths of this review include the wide search ensuring all available literature was gathered and the detailed cost analysis. The main limitation is the relatively small number of studies available with numerical data on costings and population comparisons. Several papers had substantial recruitment numbers (n=1578 [30), but many were small (n=26 [13]), reducing the reliability. Most papers focused on ages in the range of 18-30 years. Carter-Harris L (2016) [9] recruited those aged 55-77 years, showing that although older people may be less likely to adopt newer technologies (of those over 65 years, 48% are active Facebook users compared with 64% for 50-64 year olds, 79% for 30-49 year olds, and 82% for 18-29 year olds [56]), recruitment can still be successful, reporting US $1.51 cost per participant. The expected barrier from lack of Internet access or experience in the older population is smaller than most think.

The percentage of people with access to the Internet is steadily increasing [1], and procedural methods can be put into place to prevent this misrepresentation of data [57]. Young, SD (2013) found that even 79% of homeless youths manage to access social media sites once per week [58]. Although Internet access currently remains to be a barrier, it seems to be smaller than barriers facing traditional methods and is set to improve in the future.

There is growing evidence to suggest that Facebook is a successful recruitment tool, and its use, therefore, should be considered when implementing future health research. Benefits include reduced cost, shorter recruitment periods, better representation, and improved participant selection in young and hard to reach demographics. This may spell the end for traditional methods, although currently the minor limitations of Internet access and the over representation of young white women may make its use inappropriate in some settings.