Published on 08.03.17 in Vol 19, No 3 (2017): March
Online Recruitment: Feasibility, Cost, and Representativeness in a Study of Postpartum Women
Background: Online recruitment is feasible, low-cost, and can provide high-quality epidemiological data. However, little is known about the feasibility of recruiting postpartum women online, or sample representativeness.
Objective: The current study investigates the feasibility of recruiting a population of postpartum women online for health research and examines sample representativeness.
Methods: Two samples of postpartum women were compared: those recruited online as participants in a brief survey of new mothers (n=1083) and those recruited face-to-face as part of a nationally representative study (n=579). Sociodemographic, general health, and mental health characteristics were compared between the two samples.
Results: Obtaining a sample of postpartum women online for health research was highly efficient and low-cost. The online sample over-represented those who were younger (aged 25-29 years), were in a de facto relationship, had higher levels of education, spoke only English at home, and were first-time mothers. Members of the online sample were significantly more likely to have poor self-rated health and poor mental health than the nationally representative sample. Health differences remained after adjusting for sociodemographic differences.
Conclusions: Potential exists for feasible and low-cost e-epidemiological research with postpartum populations; however, researchers should consider the potential influence of sample nonrepresentativeness.
J Med Internet Res 2017;19(3):e61
Internet-based recruitment and data collection for epidemiological research has been shown to offer benefits in terms of feasibility and accessibility (ie, low-cost, time-efficient, access to hard-to-reach populations, broad geographical/global reach) [, ]. Importantly, evidence is also accumulating that data obtained via Internet-based research methods is reliable and valid [ - ]. Research also suggests that Internet recruitment can provide a sample of young adults broadly representative of the target population of interest [ , , , ].
One specific population-group in which Internet-based research and data collection may be highly enabling, but where little research is available, is postpartum women (first 12 months after birth). Previous research investigating postpartum parents has relied largely on recruitment via hospitals and child/maternal health clinics, requiring considerable time and financial investment. However, there may be significant emerging opportunities to recruit this population more efficiently online. Internet usage in this group is likely to be high, given postpartum women are often socially isolated at home, restricted in their mobility, and time-poor [- ]. Studies successfully recruiting women planning a pregnancy (preconception) [ , ], and small samples of postpartum women [ , ], suggest that online recruitment is feasible. However, little is known about how participants recruited online might differ from postpartum women in the general population.
The current study investigates the feasibility of recruiting a large sample of postpartum women online for health research, and examines sample representativeness. Sociodemographic and health characteristics were compared between an online sample and another postpartum sample that was recruited face-to-face in the context of a nationally representative household survey.
Participants and Procedure
The Living with a Young Baby Survey
Participants were recruited during January and February 2015 for an anonymous online survey investigating women’s general postpartum health and psychological wellbeing. Online recruitment involved two processes: (1) advertisement on a popular Australian pregnancy/infant-focused website babycentre , and (2) targeted Facebook advertisements. The brief advertisements targeted women who were aged >18 years, who resided in Australia with a young baby: “ Research: Mums Wanted! Researchers are looking for Australian women with a young baby to do a brief survey ”. No incentive was provided for participation. Mental health was not mentioned, in an effort to minimize health selection. Individuals who clicked on the advertisement were redirected to the online survey, which took approximately 15 minutes to complete. The Living with a Young Baby Survey (LYBS) was completed by 1083 respondents. The LYBS was approved by The Australian National University Human Research Ethics Committee.
Household Income Labour Dynamics in Australia Study
Comparative data were drawn from waves (time-points) 11 and 13 of the Household Income Labour Dynamics in Australia Study (HILDA), as at these time-points the K-10 Psychological Distress Scale was included in the survey. HILDA is a longitudinal nationally representative household panel survey that has been conducted annually since 2001 . At baseline, the study recruited participants using a multi-stage sampling approach, sampling households within a selection of administrative areas. At baseline, 7682 households were involved, including 13,969 individual household members aged 15-95 years. Completion of the HILDA survey involves both a brief face-to-face personal interview and a paper-pencil self-completed questionnaire. The baseline response rate (66%) and individual-level reinterview rates (96% in wave 13) are comparable or superior to other national household panels around the world [ ]. The sample is not static, as new participants enter study households and a significant top-up sample was introduced in wave 11. In addition, weights provided with the dataset enable adjustment for selection and attrition to better reflect the national population [ ]. The current HILDA analyses were restricted to postpartum women aged 18-50 years (youngest child <1 year), providing a total of 579 respondents (wave 11 n=288; wave 13 n=291).
Sociodemographic measures included location (state and remoteness), language spoken at home, age, relationship status, number of children, and level of education completed. General self-rated health was measured by asking, “In general, how would you say your own health is?” Possible responses were, “excellent”, “very good”, “good”, “fair”, or “poor” . Psychological distress was measured using the K-10 Kessler Psychological Distress Scale [ ]. The K-10 has ten items representing symptoms of psychological distress, with a total scale score ranging from 10-50. The total score was categorized into four groups: 10-15 (Low), 16-21 (Moderate), 22-29 (High), and 30-50 (Very High), consistent with previous research [ , ].
Descriptive statistics for sociodemographic and health variables from both samples are presented. The data reported for the HILDA sample was weighted by the cross-sectional population weights included in waves 11 and 13. Chi-square tests of association identified characteristics that differed between the LYBS and HILDA samples. Univariate logistic regression provided estimates of the magnitude of differences in characteristics between the LYBS and HILDA samples (odds ratios). Multivariate logistic regression (including all variables) tested whether differences in sociodemographic characteristics accounted for differences in general health and mental health between the samples. The weighted HILDA data provides the best possible comparison group available: postpartum women can be identified, self-rated health and the K-10 are reported (comparable measures to the LYBS), and the sample is representative of the Australian population.
Online recruitment from the babycentre website took place over 9 days, and 264 surveys were completed. There was no cost associated with advertising on the babycentre website. Online recruitment from Facebook took place over 4 days, and 819 surveys were completed. Information provided by the Facebook Ads Manager shows that across the 4 days the advertisement was clicked on 2647 times, and the total audience reach was 38,765. Given that 819 surveys were completed, we estimate a conversion rate of click-to-completed-survey of 30.94% (819/2647). The total cost for advertising on Facebook was Aus $448.68 (the cost per click on the advertisement was Aus $0.17), and cost per survey completion was Aus $0.55.
presents both descriptive information and results from logistic regression models. Compared to mothers in the nationally representative HILDA survey, mothers in the online LYBS were more likely to be located in the Australian Capital Territory (+4.9%), to be located in a remote or rural area (+1.5%), to speak English only at home (+15.4%), to be younger (+7.7%), to be in a de facto relationship (+7.4%), and to be a first-time mother (+16.5%). Mothers in the online survey were less likely to have not completed high school (-5.5%). Mothers in the online LYBS had poorer general health and higher psychological distress than those in the HILDA study. Multivariate logistic regression (final column in ) showed that including all sociodemographic characteristics did little to attenuate the association between poor general health (and poor mental health) and participation in the LYBS online sample (compared to the HILDA sample). Those with fair/poor general health (as opposed to excellent health) remained 2.66 times more likely to participate in the LYBS online sample, compared to the HILDA sample. Those with moderate distress (as opposed to low distress) remained 2.73 time more likely to participate in the LYBS sample; those with high distress were 2.03 times more likely, and those with very high distress were 3.92 times more likely.
The online recruitment strategy was highly time-efficient and low-cost. Over a period of thirteen days, 1083 participants were recruited for a total direct cost of Aus $448.68. The recruitment cost for the current online study is significantly lower than has been reported previously in studies conducting online recruitment [, ]. This result is likely due to the combined strategy of utilizing free website advertising and commercial Facebook advertising, and because the current recruitment approach targeted a very specific population (postpartum women).
There were several significant differences in sociodemographic characteristics between the online LYBS sample and the HILDA sample. One of the greatest differences was that the LYBS recruited more first-time mothers. First-time mothers may be more likely to self-select into research focused on postpartum experiences, given the experience of motherhood is new and highly salient. The higher education levels observed in the LYBS sample likely reflect greater education among social media users in general , and in the context of motherhood, those who engage with infant care and development websites. Women in the online LYBS sample had poorer general health and mental health. Previous research has similarly found that online recruitment for mental health surveys attracts a sample with higher levels of psychological distress than is reported by the general population [ ]. This finding may be due to the fact that online recruitment is more susceptible to uncontrolled snowballing as participants recruit others that they think will be interested in the research. It is also possible that reports from the general population under-represent levels of psychological distress [ , ]. Given that the poorer general health and mental health of postpartum mothers recruited online was not simply due to sociodemographic variation, further investigation is needed to understand who participates in online health research and why, and how the characteristics of this population may affect the results obtained.
The extent to which sample differences associated with recruitment method will impact on the validity of research findings largely depends on the research question being investigated [, , ]. The current findings suggest that when the goal is to estimate the prevalence of physical and mental health problems in postpartum women, Internet recruitment will likely provide an overestimate. Alternatively, when the focus is on the relationship between exposure and illness (etiology), Hatch et al state, “representativeness of a source population is arguably not a prerequisite for either internal validity or generalizability” [ ]. Nonrepresentation is chiefly a problem when characteristics that differentiate those online from those not online (eg, in this study: primiparity, non-English speaking background, education), distort or moderate the association between exposures and outcomes of interest [ ].
This study recruited a large sample of postpartum mothers online, demonstrating both efficiency and low-cost. Online samples may over-represent first-time mothers with higher education and English-speaking backgrounds, as well as those experiencing physical and mental health problems. Unmeasured differences in survey methodology play a selection role (ie, length of survey, anonymity in the LYBS), and the findings may be unique to the Australian context. While the monetary figures clearly demonstrate that online recruitment is low-cost, we did not specifically conduct cost-effective analyses. Researchers adopting Internet methods to investigate postpartum health should consider the implications of sample nonrepresentation in relation to their specific research aims (eg, prevalence, etiology, intervention).
This work was supported by Australian Research Council Future Fellowship #FT13101444, National Health and Medical Research Council (NHMRC) Early Career Fellowship #1035803, and NHMRC Career Development Fellowship #1083311. HILDA was initiated and funded by the Australian Department of Social Services (DSS) and is managed by The Melbourne Institute for Applied Economic and Social Research. The findings and views reported in this paper, however, are those of the author and should not be attributed to either DSS or The Melbourne Institute. There have been no involvements that might raise the question of bias in the work reported or in the conclusions, implications, or opinions stated.
Conflicts of Interest
Multimedia Appendix 1PDF File (Adobe PDF File), 34KB
- Batterham PJ. Recruitment of mental health survey participants using Internet advertising: content, characteristics and cost effectiveness. Int J Methods Psychiatr Res 2014 Jun;23(2):184-191. [CrossRef] [Medline]
- Fenner Y, Garland SM, Moore EE, Jayasinghe Y, Fletcher A, Tabrizi SN, et al. Web-based recruiting for health research using a social networking site: an exploratory study. J Med Internet Res 2012 Feb 01;14(1):e20 [FREE Full text] [CrossRef] [Medline]
- Andreeva V, Galan P, Julia C, Castetbon K, Kesse-Guyot E, Hercberg S. Assessment of response consistency and respective participant profiles in the Internet-based NutriNet-Santé Cohort. Am J Epidemiol 2014 Apr 01;179(7):910-916 [FREE Full text] [CrossRef] [Medline]
- Konstan JA, Rosser BR, Ross MW, Stanton J, Edwards WM. The story of subject naught: a cautionary but optimistic tale of Internet survey research. J Comput Mediat Commun 2005;10(2):A-00. [CrossRef]
- Hatch EE, Hahn KA, Wise LA, Mikkelsen EM, Kumar R, Fox MP, et al. Evaluation of selection bias in an Internet-based study of pregnancy planners. Epidemiology 2016 Jan;27(1):98-104 [FREE Full text] [CrossRef] [Medline]
- Harris ML, Loxton D, Wigginton B, Lucke JC. Recruiting online: lessons from a longitudinal survey of contraception and pregnancy intentions of young Australian women. Am J Epidemiol 2015 May 15;181(10):737-746. [CrossRef] [Medline]
- Mishra G, Hockey R, Powers J, Loxton D, Tooth L, Rowlands I, et al. Recruitment via the Internet and social networking sites: the 1989-1995 cohort of the Australian Longitudinal Study on Women's Health. J Med Internet Res 2014 Dec 15;16(12):e279 [FREE Full text] [CrossRef] [Medline]
- Drentea P, Moren-Cross JL. Social capital and social support on the web: the case of an Internet mother site. Sociol Health Illn 2005 Nov;27(7):920-943 [FREE Full text] [CrossRef] [Medline]
- Maloni JA, Przeworski A, Damato EG. Web recruitment and internet use and preferences reported by women with postpartum depression after pregnancy complications. Arch Psychiatr Nurs 2013 Apr;27(2):90-95. [CrossRef] [Medline]
- McDaniel B, Coyne SM, Holmes EK. New mothers and media use: associations between blogging, social networking, and maternal well-being. Matern Child Health J 2012 Oct;16(7):1509-1517. [CrossRef] [Medline]
- Huybrechts KF, Mikkelsen EM, Christensen T, Riis AH, Hatch EE, Wise LA, et al. A successful implementation of e-epidemiology: the Danish pregnancy planning study 'Snart-Gravid'. Eur J Epidemiol 2010 May;25(5):297-304 [FREE Full text] [CrossRef] [Medline]
- Wise L, Rothman KJ, Mikkelsen EM, Stanford JB, Wesselink AK, McKinnon C, et al. Design and conduct of an Internet-based preconception cohort study in North America: Pregnancy Study Online. Paediatr Perinat Epidemiol 2015 Jul;29(4):360-371 [FREE Full text] [CrossRef] [Medline]
- BabyCenter. 2017. URL: http://www.babycentre.com.au [accessed 2017-02-25] [WebCite Cache]
- Melbourne Institute of Applied Economic and Social Research. 2002. Household, Income and Labour Dynamics in Australia (HILDA) survey annual report 2002 URL: https://melbourneinstitute.com/downloads/hilda/Annual_Report/A_Report_2002.pdf [accessed 2017-02-25] [WebCite Cache]
- Melbourne Institute of Applied Economic and Social Research. 2011. Household, Income and Labour Dynamics in Australia (HILDA) survey annual report 2011 URL: https://melbourneinstitute.com/downloads/hilda/Annual_Report/A_Report_2011.pdf [accessed 2017-02-25] [WebCite Cache]
- Watson N. HILDA Project Technical Paper Series No. 2/12. 2012 Dec. Longitudinal and Cross-sectional Weighting Methodology for the HILDA Survey URL: http://www.melbourneinstitute.com/downloads/hilda/bibliography/hilda_technical_papers/htec212.pdf [accessed 2017-02-25] [WebCite Cache]
- Fayers P, Sprangers MA. Understanding self-rated health. Lancet 2002 Jan 19;359(9302):187-188. [CrossRef] [Medline]
- Kessler R, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002 Aug;32(6):959-976. [Medline]
- Australian Bureau of Statistics. ABS, Information Paper: Use of the Kessler Psychological Distress Scale in ABS Health Surveys, Australia, 2007-08. 2008. URL: http://www.abs.gov.au/ausstats/abs@.nsf/mf/4817.0.55.001 [accessed 2017-02-25] [WebCite Cache]
- Andrews G, Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10). Aust N Z J Public Health 2001 Dec;25(6):494-497. [Medline]
- Morgan A, Jorm AF, Mackinnon AJ. Internet-based recruitment to a depression prevention intervention: lessons from the Mood Memos study. J Med Internet Res 2013 Feb 12;15(2):e31 [FREE Full text] [CrossRef] [Medline]
- Perrin A. Pew Research Centre.: Pew Research Centre; 2015. Social Media Usage 2005-2015 URL: http://www.pewinternet.org/2015/10/08/social-networking-usage-2005-2015/ [accessed 2017-02-25] [WebCite Cache]
- Slade T, Johnston A, Oakley Browne MA, Andrews G, Whiteford H. 2007 National Survey of Mental Health and Wellbeing: methods and key findings. Aust N Z J Psychiatry 2009 Jul;43(7):594-605. [CrossRef] [Medline]
- Woolhouse H, Gartland D, Perlen S, Donath S, Brown SJ. Physical health after childbirth and maternal depression in the first 12 months post partum: results of an Australian nulliparous pregnancy cohort study. Midwifery 2014 Mar;30(3):378-384. [CrossRef] [Medline]
- Nohr E, Olsen J. Commentary: epidemiologists have debated representativeness for more than 40 years--has the time come to move on? Int J Epidemiol 2013 Aug;42(4):1016-1017. [CrossRef] [Medline]
- Rothman KJ. Six persistent research misconceptions. J Gen Intern Med 2014 Jul;29(7):1060-1064 [FREE Full text] [CrossRef] [Medline]
|DSS: Department of Social Services|
|HILDA: Household Income Labour Dynamics in Australia Study|
|LYBS: Living with a Young Baby Survey|
|NHMRC: National Health and Medical Research Council|
Edited by G Eysenbach; submitted 09.03.16; peer-reviewed by E Hatch, A Barrera, E Nelson, R Nosheny; comments to author 28.06.16; revised version received 18.08.16; accepted 23.11.16; published 08.03.17
©Liana S Leach, Peter Butterworth, Carmel Poyser, Philip J Batterham, Louise M Farrer. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.03.2017.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.