It can be very difficult to meet recruitment goals in health research, particularly trials of interventions. Many studies fail to recruit the targeted number of participants or experience delays in participant recruitment [1]. A number of strategies have been proposed to improve recruitment rates [2]. Among these, Internet-based recruitment is growing in popularity, particularly in health survey research [3] and in interventions to improve physical health, such as smoking cessation interventions [4,5]. This is increasingly feasible given the high levels of access to the Internet in developed countries [6]. Internet-based recruitment has several advantages, such as the ease of reaching people from a wide geographic area including locations remote from researchers. Anonymous participation is possible, which is useful for highly sensitive topics such as illicit drug use [3] and sexuality [7]. It can also be easier to recruit large samples cost-effectively [4].

Most research on the cost of Internet-based recruitment has been published in the area of substance use and smoking cessation [4,5,8-11]. Costs per completed survey or enrolled participant have varied across (and within) these studies for a variety of reasons, such as different online advertising types (eg, Facebook, Google search, and banner advertising), the percentage of eligible participants who clicked on the ad, and the use of incentives to participate. Ramo et al [5] evaluated the cost of recruiting young adult smokers in the United States to complete an online survey about substance use through text advertisements and banner advertisements on social networking and lifestyle websites, as well as a separate study using Facebook advertising [11]. Text and banner advertisements cost an average of US $43 per completed survey, but Facebook advertising (targeted to sociodemographic characteristics and substance keywords in user profiles) cost an average of US $4.28 per completed survey. Recruitment through Google advertising was also cost-effective for several smoking cessation interventions, including 2 randomized controlled trials [4,8,9]. Google Search advertisements cost on average US $5 to $8 per registrant to QuitNet [4]. This was far less expensive than banner advertising on popular websites with broad reach, which cost up to US $476 per participant and had fewer registrants to the program. Similarly, Gordon et al [9] found an average cost per participant of US $6.70 through Google advertising, which was much less expensive than a media campaign (US $92) and newspaper advertising (US $115). Buller et al [8] found a higher average cost per participant from Google advertising (US $41), but it was the least expensive advertising type in their study, comparing favorably with US $56 for distribution of printed promotional material and US $134 for Quit-Line phone screening.

Despite the growth in Internet-based interventions for mental health problems, such as computerized cognitive behavior therapy [12], Internet-based recruitment to mental health studies is uncommon [13,14]. Evaluations of these interventions have often used traditional offline recruitment methods, such as flyers, newspaper advertising, and sourcing participants from users of health services. Recruitment problems are amplified in preventive medicine where it can be especially difficult to recruit participants to preventive trials targeting individuals at risk rather than those with a diagnosed condition (who may already be in contact with health professionals). In addition, these studies typically require large sample sizes. Trials that do not recruit participants from a health service often use media releases to advertise the study or screen members of the public through mail surveys to determine whether they meet study eligibility criteria [15].

More than a quarter of Internet users search online for information about depression, anxiety, stress, or mental health issues [16]; antidepressants are the second most searched-for treatment on WebMD [17]. Therefore, targeting individuals who use the Internet to seek information or support for mental health problems could be a useful method of recruiting participants to Internet-based mental health interventions. However, there are few detailed accounts of recruitment experiences to guide mental health researchers. Such information is critical to trial planning because recruitment success will determine the feasibility of a study, and both traditional and Internet recruitment methods can consume a substantial part of trial budgets.

We recently developed an Internet-based intervention (Mood Memos) to improve depressive symptoms in adults. The initial Mood Memos trial targeted people with mild symptoms who were at risk for depressive disorders. Thus, we did not have a geographically or service-defined population from which to sample (eg, a mental health clinic). Therefore, we decided to investigate the potential of recruiting participants solely through Internet sources. Internet-based recruitment was feasible because the intervention was completely Web-based and automated, with Web-based questionnaires used to assess eligibility. This meant that individuals who had turned to the Web for information or support about depression could seamlessly sign up to the study and start receiving the intervention immediately. Furthermore, Internet-based recruitment enabled access to a larger pool of potential participants than was possible if recruitment was restricted to our geographic location (Melbourne, Australia). This was important because the study required a large sample size to detect the predicted small effect size associated with a low-intensity preventive intervention. We explored a variety of Internet-based recruitment techniques. In this paper, we document our experience with Internet-based recruitment to a mental health intervention and the lessons we learned.

The Mood Memos study was a randomized controlled trial (ACTRN12609000925246) testing whether self-help behaviors for depression could be improved by promotional messages sent by email [18,19]. Participants received a series of automated Mood Memo emails over a 6-week period, which encouraged the use of effective self-help strategies endorsed by experts. These emails were found to reduce depression symptoms and psychological distress relative to control emails that provided information only [18]. The study was an indicated prevention trial, with a minimum of 800 participants required to have adequate power to detect the predicted small effect size. Allowing for dropouts, we set a target recruitment size of 1200 participants.

The study was open for recruitment between February 2010 and March 2011. Participants joined the study by visiting the Mood Memos website [20], and undergoing screening for subthreshold depression. There were no incentives given to participate. The website was established for the express purpose of recruiting to the study and included only the screening questionnaire, information about the study, and links for immediate help from other sources. The Patient Health Questionnaire depression scale (PHQ-9) [21] was used to screen for depression. It can be scored as a continuous measure of depression severity or by using a diagnostic algorithm to make a probable diagnosis of major depression. The frequency over the past 2 weeks of each of the 9 criterion A symptoms of a major depressive episode as defined by the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) was assessed on a 4-point Likert scale (0=not at all; 3=nearly every day). Total scores range from 0 to 27, with cutpoints of 5, 10, 15, and 20 representing mild, moderate, moderately severe, and severe levels of depressive symptoms, respectively [22]. Because we targeted participants with subthreshold depression symptoms (clinically relevant depressive symptoms at levels that did not meet the criteria for major depression), rather than use a cut-off score, we included participants with 2 to 4 symptoms of depression [23] experienced most of the time for 2 or more weeks, which had affected work, home, or social functioning. Other inclusion criteria were aged 18 years or over; not receiving treatment for depression from a health professional (not including maintenance antidepressant medication for 6 months or more); a resident of Australia, New Zealand, the United Kingdom, Ireland, Canada, or the United States; and had access to the Internet at least weekly. Once participants were screened and found to be eligible, they provided a name and email address. A hyperlink to the baseline questionnaire package was then sent to this email address. Once the baseline assessment was complete, participants were randomized to condition and immediately sent the first of 12 Mood Memo emails. The trial was approved by the University of Melbourne Human Research Ethics Committee (HREC 0931313).

Our intention was to recruit participants from a variety of online sources, and although we initially specified some recruitment sources, we were flexible in exploring new online recruitment opportunities. We trialed a range of recruitment techniques, including search engine advertising, Facebook advertising, emails to personal and professional networks, posts in forums and online noticeboards, submitting the website to website directories, distributing an online press release, and promotion through relevant websites. Advertising costs are reported in Australian dollars (AUD); during the advertising period, AUD $1 ranged between US $0.98 and $1.02 [24].

The Mood Memos study demonstrated that it is possible to recruit a large sample to a randomized controlled trial of a mental health intervention purely through Internet-based sources. We were able to meet our ambitious recruitment target over a period that was only marginally longer than planned. Participants were recruited largely through a combination of Google advertising and promotion via the online networks of mental health organizations and websites. Recruiting participants to randomized controlled trials can be very difficult, and many trials do not reach their target sample size [2]. Online recruitment can potentially reach a wider pool of potential participants more cost-effectively than traditional techniques, such as media advertising. However, promoting the study online was not without difficulty. Google advertising involved a steep learning curve and much trial and error in working out the optimal combination of ads and keywords that competed well against other advertisers, were affordable, and were searched for frequently enough to be worthwhile. Researchers using Google should be aware that establishing a campaign is not sufficient for success, and that monitoring performance during the recruitment period is critical. Changes and refinements will almost certainly be needed in the course of a recruitment campaign. In comparison to Google, many other recruitment techniques, although free, were time-consuming and less effective.

Despite recruiting a large sample, recruitment was constrained by a number of factors, including the nature of the intervention, trial design, and eligibility criteria. For example, some recommended techniques in website promotion, such as search engine optimization, had limited application for the Mood Memos website. Search engine optimization involves designing websites so that users can find them easily through search engines such as Google. Google often updates its algorithms that determine website rankings in search results. Current recommendations by experts in the field are to provide quality content, rather than rely on many incoming links from other websites [42]. Because the Mood Memos website was designed to have minimal content (just the depression-screening questionnaire, information about the study, and links to resources for more help), other techniques to reach Internet users were essential. The choice of control intervention may also have increased the difficulty of recruiting participants to the study. To keep participants blind to which condition was the control, the promotion of the study could not solely focus on self-help or coping with depression, although this would probably have generated more interest from potential participants. Using a wait-list control may have made recruitment easier because the study could have been promoted as a way of learning techniques to cope with depressive symptoms. Instead, the study was promoted in general terms as a way of receiving expert information and advice about depression. Furthermore, targeting people at risk of major depression was difficult. Experience in the Netherlands has shown that there is minimal uptake of indicated preventive interventions, even when they are available at little or no cost [43]. Part of the reason for this may be that individuals may lack self-awareness or have insufficient mental health literacy to understand that their distress could be an early sign of depression. Much of the delay in seeking professional help for depression is due to a lack of problem recognition [44]. The difficulty of recruiting participants to the Mood Memos study from websites, forums, and email groups dedicated to mental health problems may have been because users of these had identified that they had depression, but their symptoms were too severe to meet admission criteria. This may have been why Google advertising targeting people who thought they might be depressed was a superior method of recruiting participants to the study. Unfortunately, Google advertising is not free, but an advantage is that it can occur in the background; it needs minimal attention once properly established.

Our experience also highlighted the importance of undertaking formal usability tests of the website sign-up process. Although we tested it informally, after launching the website we discovered that some participants could not find the sign-up button because it was below the plain language statement. Our experience indicates that there may be tension between best practice for Web usability and the current practice of obtaining informed consent by providing a long and detailed text-based information sheet to participants. Over the years, printed information sheets have increased in length, becoming more comprehensive to meet ethics requirements. Yet this increase in completeness sacrifices comprehension because longer forms are less likely to be read and understood [45]. These issues are compounded when information sheets are presented online because less text is read online compared to print, and users may miss information or instructions on how to participate if they have to scroll down to view it [46]. These issues are worth considering to avoid wasting effort attracting visitors to your website only to lose them during the sign-up process.

Internet-based recruitment does have limitations. It was particularly suitable for our study because we did not require any face-to-face assessments of participants during the study. This may not be feasible for many evaluations of mental health interventions, even when Internet-based, because diagnostic psychiatric interviews are usually preferred to self-report assessments. However, other researchers have combined Internet-based recruitment with face-to-face assessments in their research [34]. Internet-based recruitment may also lead to low rates of participation from groups that are less likely to use the Internet, such as older adults, the less educated, and those with low incomes [47]. However, this does not necessarily need to be the case because reaching samples that are more representative is possible through demographic targeting with Facebook [34]. In addition, certain kinds of Internet advertising may lead to greater participation from groups less likely to participate through traditional recruitment techniques (eg, men and ethnic minorities) [4].

Combining Internet-based recruitment with Internet-based assessment poses challenges to research integrity, including the potential for multiple, fraudulent enrollments. This can be problematic if there are incentives to participate [48]. Although we found Internet-based recruitment effective in reaching individuals with subthreshold depression who wanted to improve their mental health, others have found it failed to fulfill its promise [49] and is often more challenging than first anticipated [7,50]. There is limited knowledge about which factors contribute to successful Internet-based recruitment, but it may be important to use techniques that build rapport and gain the trust of potential participants, and to balance broad exposure with appropriate targeting and tailoring of recruitment messages [3]. If direct rapport is not feasible, then derived rapport may be sufficient. This can be achieved by recruiting participants through trusted people or organizations that have an existing link with potential participants. In our study, the effectiveness of recruitment through the University of Melbourne, beyondblue, and MHFA may indicate the importance of this factor.

Internet-based recruitment is becoming an increasingly important mode of recruitment; however, there are limited data available to guide researchers on the best recruitment strategies. Our study showed that several techniques, including Google advertising, were successful in recruiting participants to an online depression intervention. Our results suggest that Internet-based recruitment to mental health interventions is feasible and can be relatively affordable.