Adult cohort studies (such as the UK Biobank, recruiting 500,000 participants and costing approximately £250 million (US $349 million) to date [1]) have demonstrated the power of mega-cohorts to transform the speed, precision, and capacity for high-value new knowledge for health and health care delivery. Unfortunately, high-profile early life initiatives of similar size and ambition, such as the US National Children’s Study and UK Life Study, were withdrawn despite £0.8 billion (US $1.2 billion) and £38 million (US $59 million) funding, respectively [2,3], in large part because they stumbled at the first hurdle of engagement and uptake. Others, though successful in recruitment, have had substantial attrition over time [4]. Thus, a limited science of engagement and retention poses a critical hurdle to such studies in meeting their vision of advancing human health.

Engagement is defined as “the extent to and manner in which people actively use a resource and has been operationalized as a multistage process involving the point of engagement, a period of sustained engagement, disengagement, and reengagement” [5]. Many factors may influence the engagement process at different time points. In a research study, indicators of poor engagement may include low initial uptake from the first point of contact or reduced interaction over time, in some cases leading to complete disengagement or dropout. Engagement strategies have been developed to enable cohort studies—both observational and interventional—to meet their aims (eg, improving health behaviors and outcomes) by allowing regular, sustainable engagement with large numbers of participants via remote or digital-only studies [6].

e-Engagement incorporates the participation, recruitment, and retention of participants through digital platforms. Factors that may improve participant e-engagement include its technical features, content, frequency of waves, and engagement-facilitation interventions (EFIs) [7]. User characteristics and digital platform features should also be considered. Ritterband et al [8] simplified this in their internet intervention model, hypothesizing that behavior change is influenced by the stepwise progression of environmental factors, support, and website characteristics affecting adherence, which then affect behavior change (ie, sustained participant engagement) through various mechanisms of change. Thus, maximizing e-engagement can improve the efficiency of research processes and reduce both administration costs [9] and the validity and power costs of significant and systematic nonuptake and attrition [10] in major studies.

Given the expense of longitudinal cohort studies, effective strategies that engage and retain cohort participants are critical to the integrity of research outcomes [11,12]. The retention of study participants is vital to ensure the power and internal validity of longitudinal research [13-15], whereas participant engagement is important for evaluating the efficacy and generalizability of the program under study. A review of randomized controlled trials [16] suggests that delays in participant recruitment or high dropout rates postrandomization may lead to uncertainty in treatment effectiveness and possibly confound results. For example, in the case of technology-based intervention studies, the technology may change over time if recruitment is prolonged, potentially leading to artifacts or differential effects on treatment outcomes. Proposed retention strategies involving (1) contact and scheduling methods, (2) visit characteristics, (3) study personnel, (4) nonfinancial incentives, (5) financial incentives, (6) reminders, (7) special tracking methods, (8) study description, (9) benefits of study, (10) reimbursement, (11) study identity, and (12) community involvement [17,18] may influence participant retention rates. However, there is limited experimental evidence and data for the in-depth exploration of retention strategies and their implementation.

With the recent growth in experimental research on the optimization of digital methods in longitudinal cohort research studies, there is a need for the literature to be collectively synthesized at a pace reflecting the rapid evolution of technology.

The objective of this review is to provide an overview of strategies that enhance engagement, participation, and retention rates in large-scale digital contact studies, comparing digital methods with alternative (digital and nondigital) methods. This work has been undertaken as part of the design of the forthcoming Generation Victoria (GenV) study [19]. GenV is a whole-of-state birth and parent cohort being planned in the state of Victoria, Australia. After initial face-to-face recruitment, the majority of contact with study participants will be via digital methods. The findings of this review are expected to inform GenV and other very large birth cohort studies in planning.

The protocol of this rapid review was registered with PROSPERO (International Prospective Register of Systematic Reviews; registration number CRD42020155430). We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement to report our systematic review [20].

In the context of the administration of large-scale digital contact cohort studies, we investigated the following research questions:

We used the following definitions throughout:

Four authors (MW, JN, SAC, and YW) developed the search strategy and refined the searches with an experienced librarian. The search queries used to retrieve our systematic reviews and meta-analyses are presented in Multimedia Appendix 1. Literature searches were performed in PubMed and Ovid MEDLINE databases using both MeSH and free-text words. The results from each search engine were downloaded into an EndNote (Clarivate Analytics) reference library and saved in Covidence (Veritas Health Innovation Ltd). Duplicate studies across the combined groups were removed. We also consulted experts and manually searched for relevant studies.

Two authors (selected from JN, YW, and LC) independently screened each paper title and abstract for relevance. The full text of the remaining papers was independently screened by two of the authors (selected from JN, YW, SAC, and LC). Any disagreements were resolved by consensus. To ensure a standardized process for our review, the author’s pilot-tested titles and abstracts, and full-text screening with a sample of papers. This information helped refine the inclusion and exclusion criteria.

For initial title and abstract screening, we excluded the following publications: the primary focus (participant) was adults ≥50 years or children as the primary respondents; publications not written in English; and publications with full text not accessible through the University of Melbourne library.

Additional criteria for the full-text review screening were not reporting our outcome metrics of engagement, participation, or retention; focusing on low-income countries; and focusing on rare or uncommon conditions such as HIV or cancer.

A data extraction template was developed and piloted by the authors (JN, YW, SAC, and LC) in 3 reviews. The template contained general review information (author and search dates), characteristics of included studies (number of relevant studies, study designs, health topics, population age, and geographic area), e-engagement, participation, and retention promoting strategies, the methodological quality of systematic reviews, and a summary of review results and conclusions.

Data were extracted independently by 2 of the authors (selected from JN, YW, SAC, and LC), and any discrepancies were resolved by consensus.

A meta-analysis was not performed because of the heterogeneity of intervention types, study designs, study populations, and outcome variables among the included studies. Instead, a narrative summary of the findings across studies was created based on study outcomes (ie, participant engagement, participation, completion, and retention) and strategies promoting these study outcomes.

Two authors (JN and YW) independently assessed the quality of the included review methodology using AMSTAR-2 (Assessing the Methodological Quality of Systematic Reviews 2; a measurement tool to assess systematic reviews) [21]. The appraisal tool of AMSTAR-2 included 16 domains: whether there was a description of the PICO (population, intervention, control group, and outcome) components in the research questions and the inclusion criteria; the protocol of systematic review or meta-analysis; study design rationale; the literature searching strategy; study selection; data extraction; specific details of inclusion and exclusion criteria; adequate detail of the included studies; bias risk assessment of the included studies; the funding sources; appropriate statistical methods; the impact evaluation of the individual study’s risk of bias (RoB); the explanation of RoB in individual studies; a satisfactory explanation for any heterogeneity; adequate investigation of publication bias; and potential conflicts of interest. The answer options for the AMSTAR-2 were yes, partial yes, and no. Yes denoted a positive result. No represented that there was not enough information about the domain. Partial yes represented that it partially adhered to the standard. Discrepancies were resolved by consensus.

The search strategy yielded 1080 systematic reviews. An additional 13 reviews were identified through a manual search of publications’ reference lists. Following the removal of duplicates, 1071 publications were screened. The title and abstract screening excluded 907 reviews. A total of 164 articles underwent full-text review, of which 19 publications [22-40] met the inclusion criteria. Figure 1 summarizes the search and screening process presented in the PRISMA format.

On simple summing, the 19 reviews contained 437 studies and more than 556,000 participants (4 studies did not report the number of participants). Some studies and therefore participants are included in more than one review. Given that we aimed to identify strategies that may influence the outcomes of interest rather than synthesize an overall estimate (as per meta-analytic techniques), we did not see an overlap as problematic.

The characteristics of the included reviews are summarized in Table 1. Most reviews contained studies of varying designs, spanning quantitative and qualitative analyses.

The 19 systematic reviews included studies conducted in both research (8 reviews) and clinical or health care (11 reviews) settings. Of these reviews, 4 examined young people and adults with mental disorders, anxiety, and depressive symptoms. Others examined engagement, participation, and/or retention in digital contact studies among perinatal and postpartum women, patients with chronic diseases, parents in neonatal intensive care units, human papillomavirus vaccine uptake in young children, vaccinations in adults, pregnant women, and uptake of preschool vaccinations.

We found 4 reviews [22,25,31,32] reporting on several technical design features to aid engagement, participation, and retention, including financial incentives (including gifts), digital pushes (SMS text message alerts), voice notifications, and email or SMS text message reminders and studies’ technical feasibility and usability (ie, informed consent). Email or SMS text message reminders and SMS text message notifications were reported in 4 reviews [31,32,36,37] as the most commonly used technical design feature to improve participation and completion rates. Two reviews reported the use of email or SMS text message reminders and voice notifications [36,37] to enhance study participation. Robotham et al [36] compared zero, one, and multiple SMS text message notifications and voice notifications.

The following EFIs were reported by 5 reviews [24,26,28,30,35] as a means to aid uptake and participant engagement.

Feedback was valued by perinatal and postpartum women and parents in neonatal intensive care units, as reported by 4 reviews [24,28,33,34].

In the review by Merten et al [34], participants valued visual and personalized feedback, information, and tools for physical activity and dietary intake tailored to their self-monitored data during pregnancy. This feedback reinforced successes and/or offered motivational support and recommendations to achieve their goals. Ames et al [24] reported that the opportunity to offer feedback about needs, preferences, and experiences during pregnancy helped develop or improve the study intervention. In the review by Dol et al [28], parents valued a video-feedback intervention that guided them to reflect on their own successful interactions through recordings of parent-infant interaction and feedback from a video interaction guidance professional. According to Lim et al [33], many of the characteristics of DIs that postpartum women valued included feedback and goal setting. Women valued setting realistic goals through video consultation with their dietitian and tracking daily weight, exercise, and blood glucose levels in a web-based intervention, consistent with known key strategies for behavior change.

Overall, 63% (12/19) of reviews reported the effectiveness of e-engagement or participation promotion strategies. Table 2 summarizes the effectiveness of the various e-engagement, participation, and retention strategies reported in the reviews. Most findings were reported as relative rather than absolute differences, where numerical syntheses were provided (refer to Multimedia Appendix 2 [22,23,25,27,29,31,32,34,36,37,39,40] for further details on the answered research questions and strategies).

The assessment of the 16 items of AMSTAR-2 from each included review is demonstrated in Multimedia Appendix 3 [22-40]; 11 systematic reviews were rated as critically low [22,23,25,26,30-36] and 8 [24,27-29,37-40] were rated as low quality. Items 7 and 10, as indicated in Table 3, were rated as particularly low quality. All systematic reviews except 1 [29] reported potential sources of conflicts of interest, including any funding they received for conducting the review, but no review reported on the sources of funding for the studies included in the review. It is important to note that AMSTAR-2 does not evaluate the quality of the primary studies. Its objective is to evaluate the methodological quality of the systematic reviews, considering how well the systematic review was conducted (eg, literature searching and data pooling). Therefore, if a systematic review included primary studies with a high RoB but the review itself was well conducted, the review tended to be rated as high quality.

In Table 3, we detailed the overall confidence in the results of each included systematic review. Reviews performed poorly with respect to (1) reporting sources of funding for included studies (0/19, 0%), (2) adequately investigating publication bias (small study bias) and discussing its likely impact on the results of the review (4/19, 21%), and (3) providing a list of excluded studies and justifying their exclusions (6/19, 32%).

Out of the 19 reviews, 10 (53%) accounted for such bias in individual studies when interpreting and discussing the results of the review and 9 (47%) reviews used a satisfactory technique for assessing the RoB in individual studies included in the review. For reviews that used a satisfactory technique for assessing RoB, most [23,26,27,29,35,36] suggested that e-engagement, participation, and retention promotion strategies were effective.

This study reviewed the current state of research comparing alternative strategies to maximize participant engagement, participation, and retention in large digital studies (as held in narrative and systematic reviews). We explored EFIs and study design features that aid engagement, participation, and retention. For reviews that met the inclusion criteria, there was substantial heterogeneity across studies in terms of e-strategies.

Most reviews show that e-engagement and participation promotion strategies are effective, which is promising. However, these reviews canvassed relatively few experimentally tested strategies, suggesting that the myriad of alternative strategies that may have been tested have not yet been the subject of reviews. Many studies have reported these features as a secondary goal of objectives such as adherence to therapy rather than as a primary goal in and of itself. From the 19 reviews, few contained very large digital studies that directly compared alternative strategies to examine impacts on engagement and retention; this may reflect the small number of such mega-studies worldwide. However, contributing reviews contained multipurpose (observational and interventional) cohort studies conducted in clinical and research settings. Motivation for study engagement, participation, and retention may differ somewhat between observational and clinical intervention studies where the participant can potentially directly benefit from participation, although the successful strategies make sense and at face value seem likely to generalize to both settings. In the absence of more tailored evidence, engagement strategies successful in intervention studies may be the best evidence we have, though they should be cautiously applied.

In the context of technical study design features, evidence suggests that using email or SMS text message reminders and voice notifications enhanced participant attendance to health care clinics. Although promising, these results should be interpreted with caution given the short duration of the e-intervention and reliance on self-reported medication adherence measures. Future studies need to determine the features of text message interventions that improve success and appropriate patient populations, sustained effects, and influences on clinical outcomes.

Human facilitation or support was important in influencing the uptake, engagement, and outcomes of digital technologies [26]. As illustrated by completion modules and completer rates, the larger effect sizes found in guided interventions suggested increased intervention adherence.

Reviews examining the effectiveness of e-engagement, participation, and retention interventions in the context of a health care intervention (rather than a cohort study) suggested the following:

Overall, no specific e-intervention strategy was identified as being superior. However, more interactive methods of delivery, such as videos and regular e-therapist contact for training, (1) improve adherence, (2) increase completion rates, and (3) improve fidelity. Further research is needed to understand the strategies that improve retention in longitudinal studies.

We limited our search to systematic reviews published between 2012 and 2019. These reviews should give good reach into source studies during the preceding decade while encompassing the rapid evolution of technology and the explosion of digital methods in this period and thus relevant to the new studies of the 2020s. However, we acknowledge that this is an arbitrary choice. Although all of the literature sourced reported on studies using partial or fully digital contact with participants, much was in the context of interventions and may not be wholly applicable to observational cohort studies. Nonetheless, those strategies found to be successful in interventional settings seem worthwhile to explore in cohort studies. We obtained low-quality ratings for some systematic reviews. We also note that although high engagement and retention are the best strategies to obtain powerful representative data sets, statistical techniques such as multiple imputation are vital adjuncts.

Although all studies want to maximize the recruitment and retention of study participants, the best methods to do this, particularly in digital settings, are understudied. This review adds to the small but growing literature on methods for optimizing engagement and participation in digital contact cohort studies. Evidence-based recruitment and retention methods are particularly important to the success of the next generation of very large birth cohorts, which are very expensive but have low funding per participant and require high retention throughout decades despite participants having no or very little in-person contact with the study team. Ideally, such studies will not only use existing evidence-based methods but will also build on experimental studies of alternative engagement and retention methods to build the evidence base of the science of science.