This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
There are many challenges in recruiting and engaging participants when conducting research, especially with HIV-positive individuals. Some of these challenges include geographical barriers, insufficient time and financial resources, and perceived HIV-related stigma.
This paper describes the methodology of a recruitment approach that capitalized on existing online social media venues and other Internet resources in an attempt to overcome some of these barriers to research recruitment and retention.
From May through August 2013, a campaign approach using a combination of online social media, non-financial incentives, and Web-based survey software was implemented to advertise, recruit, and retain participants, and collect data for a survey study with a limited budget.
Approximately US $5,000 was spent with a research staff designated at 20% of full-time effort, yielding 2034 survey clicks, 1404 of which met the inclusion criteria and initiated the survey, for an average cost of US $3.56 per survey initiation. A total of 1221 individuals completed the survey, yielding 86.97% retention.
These data indicate that online recruitment is a feasible and efficient tool that can be further enhanced by sophisticated online data collection software and the addition of non-financial incentives.
Recruiting and retaining HIV-positive research participants is a crucial yet challenging endeavor [
In addition to these HIV-specific challenges, there are major impediments and compromises associated with research in general [
Due to the near ubiquity of the Internet [
Therefore, if perceived stigma, cost of transportation, and physical distance are potential deterrents for HIV-positive individuals to participate in research, and time, geographical, and resource constraints are among the general difficulties of research, then Internet-based resources may be a logical solution to overcome many of these barriers. Thus far, the literature has indicated that online social media is ripe for researchers to use as a tool for recruitment [
We conducted a cross-sectional survey using online social media to recruit HIV-positive individuals. The purpose of the parent study was to examine the barriers to and facilitators of adherence to antiretroviral therapy and to describe the use of mobile telephones and other technologies to improve adherence among HIV-positive individuals living in the United States. Here, we describe the methods used for participant recruitment using online social media.
We implemented a campaign approach where participants were recruited through online social media such as Facebook, Twitter, and other social media platforms, such as LinkedIn, Craigslist, and Tumblr, from May 7 to August 31, 2013. The online survey was programmed using Qualtrics Research Suite (Qualtrics, Provo, UT), an online survey tool that allows researchers to build, distribute, and analyze online surveys in real time. The survey contained 112 items on demographics, HIV clinical outcomes, and use of technology. The University of California, San Francisco (UCSF) Committee on Human Research approved this study in April 2013.
A schedule was devised by the research assistant to regularly update and maintain these social media sites for a total of 8 hours per week (20% of full-time effort).
We used 4 methods to recruit participants on Facebook: paid advertisements (ads), fan page, personal messages, and postings in groups.
A Facebook ad was created that ran separately and concurrently with the rest of the research recruitment campaign to further broaden exposure to the target population. With the assistance of a Facebook representative, a daily average budget of US $40 and a total budget of US $5,000 were established to cover the 4-month recruitment period. The average maximum bid per cost per click (CPC) was set at US $3.50, which was above the Facebook suggested bid range of US $0.52-$1.09, to outcompete other advertisements targeting the same groups. These advertisements were targeted at users with self-reported interests or “liked” topics in any of the following categories: HIV/AIDS; lesbian, gay, bisexual, transgender, queer, questioning, intersex, and asexual (LGBTQQIA); men who have sex with men (MSM); HIV co-morbidities such as tuberculosis and Hepatitis C; and unprotected sex.
A wide variety of advertisement pictures were chosen to cater to the target demographic. Pictures of inanimate or abstract objects, such as the AIDS red ribbon, were used as well as pictures of people representing various age ranges, genders, and sexual identities. Since banner advertisements targeting MSM of color have been shown to increase the click-through rate [
We initiated the Facebook ad campaign with 3 advertisements but increased the number of advertisements to 5 in order to cater 2 advertisements specifically toward women and youth (18-29 years old). This decision was in response to the Facebook Ad Reports, which showed low levels of participation by women and youth. We varied ad pictures on a bi-weekly basis based on the number of Facebook ad clicks.
A Facebook fan page was created with the intention of recruiting participants and raising general awareness of the study. This was achieved by generating interesting and relevant posts for people living with HIV, which included news articles, study announcements, survey dissemination requests, pictures, videos, and HIV-related resources. Posted news articles related to HIV, medication adherence, and the use of technology as it pertained to HIV and medication adherence. On average, the research assistant updated the fan page every 3 to 4 days and posted about 2 new posts per day. A fan page description under the “About” section of the Facebook page was created to inform potential participants of the study and provide a link to the survey. A brief description of the study investigators and the organization was also included to build rapport and credibility with the audience. The research assistant interacted with other Facebook fan pages with similar purpose or interest by “liking” these organizations’ pages, which were found using keyword searches for HIV, AIDS, MSM, and health care.
The research assistant sent personalized Facebook messages on behalf of the study to community leaders, major HIV/AIDS organizations, and other related organizations, especially in states with low survey response rates. Survey response rates were estimated by examining Facebook Ad Reports. These messages contained information about the study, a link to the survey, and a request to repost the study link on the individual or organization’s corresponding Facebook page.
The research assistant also joined HIV-related Facebook groups to advertise for recruitment. These groups maintain Facebook pages based on real-life interests to facilitate discussions of relevant topics. Once requests to join these groups were granted, posts were made in the common thread requesting that interested participants click on the survey link.
A Twitter account was created for the purpose of recruiting participants for the study and to continue building an online network to advertise for the survey. Through the study’s Twitter account (ie, “feed”), the research assistant followed any organization affiliated with topics such as HIV, AIDS, MSM, or global health care. Individuals were also followed if they were candid about their HIV seropositive status or being MSM, an HIV advocate, or a health educator.
Tweets were sent directly to both individuals and organizations as a request to re-tweet the survey link. Hashtags (#) related to the Twitter account were used to keep track of the number of times each message was tweeted and a shorter version of the study’s URL (ie, “tiny URL”) was generated to fit within the 140-character limit of tweets. The research assistant tweeted relevant content related to HIV/AIDS or interacted with other Twitter feeds by re-tweeting interesting and applicable topics on a weekly basis.
LinkedIn groups that included content related to HIV were specifically chosen to advertise to potential study participants. Postings regarding the study were made in groups that were created for people living with HIV, HIV advocates, community workers, and other researchers. Additionally, a request was made with each posting for group members to share the survey link with other potentially interested groups or individuals.
On a weekly basis, the research assistant posted advertisements containing the survey link and basic information about the study on nationwide Craigslist groups pertaining to MSM and health. Elements from the Touro 12-Step Process [
A Tumblr blog was created and maintained with regular updates that mirrored the Facebook posts. Other blogs with content related to HIV were “liked” in order to build an audience and stimulate recruitment.
Individuals who clicked on the study Web link were directed to the Qualtrics survey where they were required to consent online before being screened for eligibility. The survey was used to screen individuals based on the following inclusion criteria: age 18 years or older, HIV-positive serostatus, and currently living in the United States. We excluded individuals who had already taken the survey by limiting the number of survey attempts to 1 per Internet Protocol (IP) address and by asking participants if they had already taken the survey. Qualtrics was programmed to end the survey if the participant did not qualify. We ensured anonymity by not collecting any identifiers or personal health information and storing responses on encrypted and password-protected servers at UCSF.
Given the anonymous nature of the Web-based survey and lack of identifiers to verify repeat and fraudulent responses, monetary incentives were not offered in order to de-incentivize multiple survey attempts. However, in order to keep participants engaged, we motivated participants by inserting 5 medically interesting facts strategically throughout the survey titled “Fun Facts”, which were accompanied by visually stimulating and relevant pictures. Examples of these fun facts include: “Did you know that we exercise at least 30 muscles when we smile?” and “Did you know that your brain uses as much power as a 10-watt light bulb?” Additionally, to spark interest, we asked participants before beginning the survey if they knew of a natural substance that can be potentially effective against HIV. Upon completion of the survey, we embedded a link to a video about how bee venom is being studied by the Washington University School of Medicine as a potential drug to treat HIV [
We consulted the UCSF Center for AIDS Prevention Studies (CAPS) Community Advisory Board (CAB), which is comprised of stakeholders from Bay Area agencies and communities with the mission of channeling community input into the CAPS research agenda, providing advice on study methods, and improving recruitment methods. Based on the recommendations of the CAPS CAB, we modified the Facebook ad pictures to exclusively use photographs of people, the Facebook ad text to be more direct and motivating, and incorporated Tumblr as an additional online social networking venue for recruitment.
Three variables were created to estimate the relative value of each of the 3 main methods of recruitment: (1) “Facebook Page Interactions” to estimate user activity on the Facebook fan page, (2) “Twitter Interactions” to evaluate the overall activity on the Twitter feed, and (3) “Facebook Ad Clicks” to assess the overall success of the Facebook ad. The number of likes, comments, and shares on the Facebook page per recruitment day were extracted from Facebook Page Insights and summed to serve as an objective measure of Facebook Page Interactions. Data collected from Twitonomy (a Twitter analytics tool) included the number of retweets and mentions per recruitment day and were used to compute Twitter Interactions. Information on the number of clicks per day for the Facebook ad was obtained from Facebook Ad Reports. Using descriptive data from Qualtrics, we collected information on the number of survey clicks per day. We used Pearson’s pairwise correlation to examine the relationship between the number of survey clicks per day and Facebook Page Interactions, Twitter Interactions, and Facebook Ad clicks per day. Correlation values of 0 to .3 were deemed weak correlation, .3 to .7 indicated moderate correlation, and >.7 were considered strong correlation. The significance level of each correlation coefficient was also estimated. A
As part of the survey, participants were also asked about their method of recruitment. These methods were categorized under one of the following categories: Facebook, Twitter, word-of-mouth, email from a listserv, and other (including LinkedIn, Craigslist, Tumblr, or other). We used 2-way frequency tables to examine the distribution of the method of recruitment based on participants’ age (categorized as 18-29.9, 30-39.9, 40-49.9, 50-59.9, and ≥60 years), race/ethnicity (White/Caucasian, African-American/Black, Latino, and other), male sex at birth, and sexual orientation (homosexual/gay, heterosexual, bisexual, and other). A chi-square test was used to assess the null hypothesis that the categorical variables are independent.
Last, to estimate research retention in the absence of monetary incentives, we compared the total number of participants who responded to the first survey question with the total number of respondents who answered the last survey question. All statistical analyses were conducted using Stata, version 13.1 (StataCorp, College Station, TX).
A total of 2034 individuals clicked on the survey Web link, averaging 18 clicks per day. Of those who clicked, 1977 people consented to take the survey, 1404 met the inclusion criteria and initiated the first question of the survey, and 1221 responded to the last question of the survey (86.97% retention). A total of 43 respondents answered affirmatively when asked if they had taken the survey before and were therefore excluded from the survey.
The study received a total of 10,006 Facebook Ad Clicks, 278 Facebook Page Interactions, and 161 Twitter Interactions during the recruitment period. Although each of these factors contributed to the total number of survey clicks per day, only the number of Facebook Ad Clicks was moderately correlated with the number of survey clicks per day based on the Pearson pairwise correlation (.52,
Number of Survey Clicks, Twitter Interactions, Facebook Interactions, and Facebook Ad Clicks in June 2013.
On average, participants spent 31 minutes to complete the survey.
A total of US $5,021 was spent on the Facebook ad (mean CPC = $0.64), which targeted a total of 14 million Facebook users and accumulated 1404 initiated surveys. Therefore, the estimated mean cost per survey initiation was approximately US $3.56. The reach of the Facebook ads included participants from 48 states plus the District of Columbia but did not include participants from either Alaska or Wyoming. From the Facebook fan page, the research assistant “liked” 490 other fan pages while 53 other organizations and individual Facebook users “liked” the study’s fan page. A total of 141 Facebook messages were sent to community leaders, organizations, and individuals requesting to have the survey link reposted.
Over the course of the recruitment period, the research assistant tweeted 572 times, received 199 Twitter followers, and followed 1092 Twitter feeds. In addition, the research assistant contacted a total of 16 LinkedIn groups, added 3 Tumblr sites, and posted on 3 Craigslist forums.
Key participant demographics and self-reported method of recruitment.
Characteristic | Total, n | Email from a Listserv | Word-of-mouth | Othera |
|
|||
|
<.001 | |||||||
|
18-29.9 | 246 | 211 (85.8) | 7 (2.9) | 15 (6.1) | 2 (0.8) | 11 (4.5) | |
|
30-39.9 | 274 | 233 (85.0) | 11 (4.0) | 21 (6.7) | 5 (1.8) | 4 (1.5) | |
|
40-49.9 | 508 | 422 (83.1) | 29 (5.7) | 36 (7.1) | 6 (1.2) | 15 (3.0) | |
|
50-59.9 | 474 | 381 (80.4) | 37 (7.8) | 33 (7.0) | 5 (1.1) | 18 (3.8) | |
|
≥60 | 402 | 256 (63.7) | 55 (13.7) | 31 (7.7) | 23 (5.7) | 37 (9.2) | |
|
Totalc | 1904 | 1503 (78.94) | 139 (7.30) | 136 (7.14) | 41 (2.15) | 85 (4.46) | |
|
.311 | |||||||
|
White/Caucasian | 995 | 802 (80.6) | 67 (6.7) | 81 (8.1) | 11 (1.1) | 34 (3.4) | |
|
Black/African American | 126 | 97 (77.0) | 14 (11.1) | 9 (7.1) | 2 (1.6) | 4 (3.2) | |
|
Hispanic/Latino | 181 | 157 (86.7) | 8 (4.4) | 9 (5.0) | 2 (1.1) | 5 (2.8) | |
|
Other | 102 | 90 (88.2) | 4 (3.9) | 3 (3.0) | 2 (2.0) | 3 (2.9) | |
|
Totalc | 1404 | 1146 (81.62) | 93 (6.62) | 102 (7.26) | 17 (1.21) | 46 (3.28) | |
|
<.001 | |||||||
|
Male | 1321 | 1102 (83.42) | 76 (5.75) | 86 (6.51) | 14 (1.06) | 43 (3.26) | |
|
Female | 77 | 39 (50.7) | 16 (20.8) | 16 (20.8) | 3 (3.9) | 3 (3.9) | |
|
Totalc | 1398 | 1141 (81.62) | 92 (6.58) | 102 (7.30) | 17 (1.22) | 46 (3.29) | |
|
<.001 | |||||||
|
Homosexual/Gay | 1208 | 1007 (83.36) | 73 (6.04) | 80 (6.62) | 11 (0.91) | 37 (3.06) | |
|
Heterosexual | 92 | 54 (58.7) | 13 (14.1) | 14 (15.2) | 5 (5.4) | 6 (6.5) | |
|
Bisexual/Other | 91 | 75 (82.4) | 6 (6.6) | 8 (8.8) | 1 (1.1) | 1 (1.1) | |
|
Totalc | 1391 | 1136 (81.67) | 92 (6.61) | 102 (7.33) | 17 (1.22) | 44 (3.16) |
aIncludes LinkedIn, Craigslist, and Tumblr.
bBased on chi-square test.
cDifferences in number of responses for each characteristic are due to missing data; participants were not “forced” to respond to each demographic question.
Our study results indicate that the unique integration of online social media recruitment, Web-based survey, and non-financial incentives is an efficient and streamlined strategy for survey research with HIV-positive individuals. The restricted resources of US $5,000 in a 4-month recruitment timeframe with 20% research assistant’s time resulted in recruiting 1404 qualified participants in 48 states and the District of Columbia, averaging 18 survey responses per day, making this an efficient solution for participant recruitment. This is similar to other studies not focused on an HIV-positive population that have demonstrated the efficiency of Facebook ads [
Even though Facebook has been used to effectively recruit HIV-positive individuals [
Although we observed a weak correlation between the number of survey clicks per day and both the Facebook Page and Twitter Interactions, these platforms were essential in establishing and maintaining a rapport with the study population and community. Such relationships were cultivated by liking and following relevant pages and groups of underrepresented populations, posting comments on walls and reposting relevant material, and sending personal messages. These techniques were particularly helpful given that the level of compliance with our requests was contingent upon the relationship and rapport between the researcher and the person or institution being contacted. For example, mass-produced messages and random posts on other Facebook pages or Twitter feeds were not as useful as personalized messages sent directly to a community leader or an HIV advocate with an established relationship. This highlights the social phenomenon known in economics as reciprocity, which correlates one’s likelihood of compliance with their familiarity with the one making the request [
Retaining participants for Internet-based studies has been known to be difficult [
Since there are more Facebook users than any other social media site, it was not surprising that most participants identified Facebook as their primary method of recruitment. Moreover, the paid Facebook ad and the selection of targeted groups likely contributed to a higher percentage of Facebook recruits. However, it is unknown why older age and non-homosexual orientations were more likely to be associated with non-Facebook methods of recruitment. Future research should further examine possible reasons for these patterns. As such, the use of online social networks other than Facebook for recruitment remains a strong area of interest as venues for future research [
The self-reported demographics elucidated in
Another limitation of our study is the possibility of duplicate responses. A study using online social media also found this issue to be problematic, especially when participants were provided a financial incentive without the use of restrictive software [
In terms of Facebook ad pictures, future research may be directed toward studying the effectiveness of different ad images by measuring the number of ad clicks and survey responses, which was a limitation of this study. Another limitation was that we were unable to verify the HIV serostatus of participants. However, given the lack of pecuniary incentives, it is unlikely that individuals would misrepresent themselves as being HIV-positive. Last, given that email from a listserv and directly contacting individuals using Facebook messages were valuable tools, it would have been beneficial for us to have asked participants about the specific listserv or the particular Facebook route (ie, Facebook ads vs personal message) in order to distinguish between various efforts.
Online social media is an indispensable tool for recruiting participants because it has the potential to be cost-effective and efficient. Social media has the unique ability to transcend barriers to study recruitment such as physical distance, transportation, and limited time and financial resources. It also affords users the added benefits of privacy and anonymity, which may ameliorate the effects of perceived stigma in difficult to reach populations. Moreover, non-financial incentives in the form of trivia are a viable alternative to monetary incentives that yields high rates of retention while minimizing the chances of duplicate and fraudulent responses. Facilitating this process using online data collection software can further reduce cost and time associated with data collection. In sum, as Internet-based research tools become more practical, researchers must embrace these methods not only to maximize efficiency but also to enhance the scientific validity and generalizability of their findings. As such, online social media, alternative types of incentives, and Web-based survey software are well poised to become staple methods of recruitment and engagement in research.
Facebook ad example.
Center for AIDS Prevention Studies Community Advisory Board
cost per click
Internet Protocol
men who have sex with men
University of California, San Francisco
The researchers would like to thank Mr Nikolai Caswell for his help in editing this manuscript. This project was supported by NIH award numbers K23MH097649, K24MH087220, and P30AI027763. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
None declared.