Eligible participants had to be adult (18 years or older), self-identified smokers who were thinking of quitting in the next 30 days, had an active Facebook account, and had not used the smoking cessation Facebook app UbiQUITous [24].

Facebook and Google AdWords advertisements were implemented simultaneously for 27 days from August 28 through September 24, 2013. All ads targeted adult smokers within the United States and were designed to be as similar as possible given the differences between Facebook and Google advertising platforms. On each platform, two identical ads were randomly displayed to users who fit the advertising parameters. Clicking on one of the ads resulted in randomization to the Internet baseline assessment (WEB), and clicking on the other ad resulted in randomization to the phone baseline assessment (PH).

A daily spending limit was set at USD $300 for all advertising. The bidding structure was set to maximize impressions in Facebook [25] and to maximize clicks in Google, with a max bid of $0.25/click. This advertising approach built on lessons learned by our research group from a previous randomized trial conducted within Facebook [24] and was consistent with each platform’s best practices at the time of the study [25-27].

Clicking on any of the ads took users to a study invitation page that provided a brief overview of the study and invited individuals to complete eligibility screening. Eligibility screening assessed gender, ethnicity, race, education, smoking status, motivation to quit, age, whether they had an active Facebook account, and whether they had used the Facebook app involved in this study. Only the last 5 questions determined eligibility; the other information was used to examine the characteristics of individuals responding to the recruitment ads.

Eligible participants completed online informed consent and provided contact information. Participants randomized to the WEB condition were immediately directed to the online baseline assessment; participants in the PH condition were informed that they would be contacted within 48 hours by a research assistant to complete the baseline assessment.

Following completion of the baseline assessment, all participants received information on how to install the study intervention, UbiQUITous, a Facebook app grounded in evidence-based smoking cessation treatment guidelines [28]. The development and implementation of the app is described in detail elsewhere [24].

The baseline survey comprised 14 questions addressing tobacco use, cessation-related cognitions and behaviors, and intensity of Facebook use [29]. It was deliberately short so as not to represent a barrier to study enrollment. At 30 days post-enrollment, participants in both arms were asked to complete a brief follow-up survey. Survey invitations were delivered via email, Facebook message, or telephone, with telephone follow-up for nonresponders. The survey assessed point prevalence abstinence (30-day and 7-day), tobacco use among those not abstinent, cessation intentions and quit methods used, and satisfaction with the UbiQUITous app. Participants were paid $15 for completion of the follow-up survey (Amazon gift certificate delivered via email).

The outcomes of interest were the proportion enrolling in the study in the WEB compared to the PH arm and of those enrolled, the proportion completing the follow-up survey at 30 days (retention) in the WEB compared to the PH condition. Secondary outcomes were recruitment volume and enrollment costs.

This study used two sources of data: advertising metrics and enrollment and follow-up data extracted from our clinical trials management system. First, we developed a CONSORT diagram to track participants from ad exposure through enrollment and retention and estimated differences in proportions at each stage using chi-square tests. Second, we estimated the advertising cost per randomized participant in the WEB and PH conditions.

Table 1 presents the flow of participants from online recruitment through follow-up. Overall, 4445 people were referred to the study from the WEB ad and 4001 from the PH ad; 12.04% (535/4445) of WEB participants and 8.30% (332/4001) of PH participants accepted the online study invitation (P<.001). There were no between-group differences in rates of eligibility or informed consent. The main reason for ineligibility was lack of intention to quit within the next 30 days (125 WEB, 57 PH). There was significant drop-off between consent and completion of the baseline survey in the PH arm, resulting in an enrollment rate of 32.7% (35/107) of PH participants versus 67.9% (114/168) of WEB participants (P<.001). The overall enrollment rate among everyone who clicked on a study ad was 1.76% (149/8446). Of the 149 participants who were enrolled, only 83 (55.7%) installed the Facebook app, with no differences between study arms (WEB: 66/114, 57.9% vs PH: 17/35, 49%; P=.33).

As shown in Table 1, 43.6% of enrolled participants completed the 30-day follow-up survey, with no differences between study arms (WEB: 49/114, 43.0% vs PH: 16/35, 46%; P=.78).

During the recruitment period, a total of $6074 (WEB $3027; PH $3047) was spent on online recruitment ads, generating 3,834,289 (WEB 2,030,253; PH 1,804,036) impressions and resulting in 8446 (WEB 4445; PH 4001) participants clicking on the ads. Advertising costs per randomized participant were $27 WEB and $87 PH.

This study used a novel randomized design to assess the impact of baseline assessment method on recruitment and retention. Our main finding was that a more intensive phone baseline assessment protocol yielded a lower rate of enrollment, equivalent follow-up rates, and higher enrollment costs compared to a web-based baseline assessment protocol. The overall enrollment rate of 2% was smaller than other trials but still in the range of 6% observed in another large web-based randomized trial [13]. Overall retention at 30 days in our study was 44%, similar to previous literature on web-based randomized trials [8] and cohort studies [30] but lower than reported in other web-based cessation studies with longer follow-up periods [13,31]. The equivalent retention rates in the web and phone arms are consistent with findings reported in a weight management study [32]. Our enrollment costs were also in line with recent studies recruiting via Facebook ads [26,27].

While our study suffered from low enrollment, differential drop-out between study arms, and low retention, these metrics were in line with those reported in other online trials [8,13]. Of interest to others conducting online recruitment, we found Google AdWords to be an ineffective recruitment strategy driven by automated shut-off by Google due to poor performance. Possible barriers to performance from the Google ads may have been the frame shift needed from an Internet search query to Facebook app installation (versus Facebook ad for a Facebook app) or the design of the ads, which attempted to keep content and pricing similar in order to directly compare the yield of Facebook to Google advertising. Researchers using online ads across different platforms for study recruitment should take these concerns into consideration when designing and implementing recruitment protocols.

One strength of our study is the randomization of participants to recruitment method using the underlying auction mechanism to place ads within Facebook and Google. Our study is one of the first to explicitly examine the impact of enrollment strategies on recruitment and retention metrics within the context of online intervention research. Future research should focus on honing advertising strategies and web-based assessment protocols to further optimize enrollment and retention.