Published on 27.05.10 in Vol 12, No 2 (2010): Apr-Jun
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/1314, first published Jul 23, 2009.
The Touro 12-Step: A Systematic Guide to Optimizing Survey Research with Online Discussion Boards
The Internet, in particular discussion boards, can provide a unique opportunity for recruiting participants in online research surveys. Despite its outreach potential, there are significant barriers which can limit its success. Trust, participation, and visibility issues can all hinder the recruitment process; the Touro 12-Step was developed to address these potential hurdles. By following this step-by-step approach, researchers will be able to minimize these pitfalls and maximize their recruitment potential via online discussion boards.
J Med Internet Res 2010;12(2):e16
A Proposed Process for Posting Surveys in Discussion Boards
The Internet, particularly online discussion boards, can be a useful and low cost instrument in recruiting participants for online surveys and data collection [, ]. Online discussion boards often provide quick access to hundreds or even thousands of participants with similar interests within a relatively short period of time [ ]. Furthermore, discussion board users generally encompass diverse geographical (often worldwide) and demographic segments of the study population universe and can be useful in facilitating and streamlining the recruitment process [ , ]. Multiple studies and a review by Krantz and Dalal have shown that web-based data collection and traditional methods (e.g. paper and pencil) result in equivalent conclusions, demonstrating the validity and reliability of online data collection for research [ , , , ].
Despite the tremendous potential, recruitment of subjects via online discussion boards may not be an easy task [, ]. A study by Koo and Skinner describes the struggles in obtaining subjects from online discussion boards including: 1) Survey postings being immediately removed by a board administrator; 2) Messages and survey links being mistaken for “spam,” and 3) Having poor visibility on the discussion forums within a few days of the initial post [ ]. As a result, the authors expressed disappointment in the small number of subjects recruited for a study. Presented here are insights and strategies to address these issues. These insights were gained from experience developing a systematic approach to successfully recruit study participants via online discussion boards.
A clinical research team at Touro University-CA College of Pharmacy in Vallejo, CA utilized various bodybuilding, weightlifting, fitness, and anabolic steroid discussion forums to recruit subjects to participate in an online research survey (Touro University-CA IRB# P-0308). Since many of the discussion boards approached require registration to gain access, an informed consent page was created to deal with potential privacy issues, to assure confidentiality and anonymity, and to provide additional information regarding the study . Clearly stated on the informed consent page were the disclosures that no individually identifiable data would be collected, that internet provider (IP) addresses would not be logged, and that all data transfer would be encrypted. After consenting to take part in the study, the strength-trained subject was allowed to start the 99-item survey which queried specific information related to exercise trends, medication utilization, behavior/psychiatric traits, and demographic variables.
Between February and June 2009, the survey was posted on over 50 different discussion boards with varying success. Encountered initially were several limitations resulting in low survey attempt and completion rates. These limitations included: 1) Postings and survey link being quickly removed by website administrators; 2) Lack of initial trust from discussion board members; 3) Lack of enthusiasm by members for participation in a survey with no apparent reward; and 4) Losing visibility of the thread and survey link when it was no longer on the first page of the discussion board forums. After encountering these hurdles during the early phases of enrollment, a systematic step-by-step (12-step) process was developed to improve the popularity and visibility of the survey link on respective sites. This 12-step approach led to a marked increase in survey attempts and completion rates. Using this 12-step method, the link was successfully posted on over 30 individual sites, resulting in over 2,250 survey attempts generated worldwide with over 1,500 individuals (518 admitted anabolic steroid users) completing the survey during a relatively short four-month window (February-June, 2009).
The Touro 12-Step Process
- Use an internet search engine (e.g. Google.com) to search for websites that have discussion boards which suit your study’s topic (e.g. “bodybuilding forums,” “weightlifting forums,” “steroid discussion boards”).
- When an appropriate website discussion board is found, determine if the discussion board has an adequate number of members/views/activity.
- Sign up as a member of that discussion board (create a user name and password).
- Look for a discussion section that is most appropriate to introduce the survey (e.g. “Bodybuilders,” “Powerlifting,” “Anabolic Steroid Discussion,” “Female Bodybuilders”).
- Create a simple yet accurate title for the thread (e.g. “Exercise Study” or “Steroid Survey”).
- Post an introduction thread that explains the research objectives and facilitates feedback/questions from the discussion board users. Include the actual name and credentials of the researcher involved, but avoid using the prefix “Dr.” as this may appear less personable. It should be emphasized: Do not include the research survey link in the first post. Website moderators and members often do not trust a researcher who is a first-time poster and may even perceive that individual as an outsider or an “intruder,” potentially altering the discussion board environment [ ]. At best, the thread may be removed—and there is a likely chance that your username and IP address will be permanently banned from the website. It is important to develop a rapport with the website members and administrators before attempting to post the survey link.
- Subscribe to the created thread so that instant e-mail notification can be received anytime a website member posts a response. Timely responses (ideally within 12-24 hours) are valuable as this demonstrates to other website members the seriousness and willingness to address their concerns.
- Only post the survey link when support of the discussion board members and moderators has been clearly established. This will increase the chances of having a high participation rate and prevent the survey link from being prematurely removed.
- Create an active and ongoing discussion. Asking board members questions and soliciting feedback will create enthusiasm about the research topic and survey.
- As days and weeks transpire, answering posts from members provides two benefits: a) continuing to increase interest in the survey and b) “bumping” or moving the survey thread back to the top of the discussion board (improving visibility of the thread).
- Be courteous. Thank participants when they make a post stating that they have completed the survey (e.g. “Thanks for supporting our survey!”). Website members appreciate the politeness and just as importantly, the “thank you” post will bring the thread back to the top of the discussion board forum (again improving visibility).
- Don’t go overboard. If there has been no activity or replies on the thread, wait at least 5-10 days before reposting (more frequent attempts to promote the survey may become an annoyance to discussion board members). Some sites may be fine with “bumping” or promoting survey participation more frequently, so pay attention and acquire a feel for the particular forum group. Try to provide value when reposting to move the thread back to the top (e.g. post progress on survey participation or provide an update on reaching the survey recruitment goal). This is especially useful towards the end of data collection to create a strong, final push.
Using the internet, especially online discussion boards, to collect survey data can be very powerful and a cost-efficient tool to promote your research survey. Over 1.5 billion individuals, roughly 23.8% of the world’s population, utilize the internet on a regular basis . To help maximize its recruiting potential, it is imperative to recognize and address potential challenges. Keys to success are to find website forums that suit the research needs, to develop a rapport with website members and moderators, to post the survey link at an appropriate time, and to strategically increase the survey link visibility through reposting and responding to website members. Finally, when the results have been compiled and are finally ready to be reported for publication, it is recommended to follow the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) guidelines to ensure quality and thoroughness [ ]. By utilizing the Touro 12-Step, researchers may be able to increase recruiting potential with online discussion boards.
Conflicts of Interest
- Rhodes SD, Bowie DA, Hergenrather KC. Collecting behavioural data using the world wide web: considerations for researchers. J Epidemiol Community Health 2003 Jan;57(1):68-73 [FREE Full text] [Medline] [CrossRef]
- Eysenbach G, Wyatt J. Using the Internet for surveys and health research. J Med Internet Res 2002 Nov 22;4(2):E13 [FREE Full text] [Medline] [CrossRef]
- Perry PJ, Lund BC, Deninger MJ, Kutscher EC, Schneider J. Anabolic steroid use in weightlifters and bodybuilders: an internet survey of drug utilization. Clin J Sport Med 2005 Sep;15(5):326-330. [Medline] [CrossRef]
- Houston TK, Cooper LA, Vu HT, Kahn J, Toser J, Ford DE. Screening the public for depression through the Internet. Psychiatr Serv 2001 Mar;52(3):362-367 [FREE Full text] [Medline] [CrossRef]
- Buchanan T, Smith JL. Research on the Internet: validation of a World-Wide Web mediated personality scale. Behav Res Methods Instrum Comput 1999 Nov;31(4):565-571. [Medline]
- Buchanan T, Smith JL. Using the Internet for psychological research: personality testing on the World Wide Web. Br J Psychol 1999;90(Pt 1):125-144. [CrossRef]
- McCabe SE. Comparison of web and mail surveys in collecting illicit drug use data: a randomized experiment. J Drug Educ 2004;34(1):61-72. [Medline] [CrossRef]
- Krantz JH, Dalal R. Validity of Web-based psychological research. In: M. H. Birnbaum (Eds.), Psychological Experiments on the Internet. San Diego: Academic Press; 2000:35-60 Academic Press.
- Koo M, Skinner H. Challenges of internet recruitment: a case study with disappointing results. J Med Internet Res 2005;7(1):e6 [FREE Full text] [Medline] [CrossRef]
- van Uden-Kraan CF, Drossaert CHC, Taal E, Seydel ER, van de Laar MAFJ. Self-reported differences in empowerment between lurkers and posters in online patient support groups. J Med Internet Res 2008;10(2):e18 [FREE Full text] [Medline] [CrossRef]
- Eysenbach G, Till JE. Ethical issues in qualitative research on internet communities. BMJ 2001 Nov 10;323(7321):1103-1105 [FREE Full text] [Medline] [CrossRef]
- Internet World Stats. URL: http://www.internetworldstats.com/stats.htm [WebCite Cache]
- Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [Medline] [CrossRef]
Edited by G Eysenbach; This is a non-peer-reviewed article. submitted 23.07.09; accepted 30.10.09; published 27.05.10
©Eric J Ip, Mitchell J Barnett, Michael J Tenerowicz, Paul J Perry. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 27.05.2010
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.