Pregnancy, delivery, and the postnatal period can be times of increased psychological distress (stress, anxiety, or depression) [1], and up to 20% of women experience depression during the perinatal period [2,3]. There is considerable evidence that psychological distress during this period has detrimental effects on maternal health and can have long-term deleterious effects on the child [4-7], as recognized as part of the Developmental Origins of Health and Disease paradigm [8,9]. In particular, there is growing evidence of intergenerational transmission of poor mental health in utero and the first years of life through these pathways [10]. Therefore, cost-effective, accessible interventions that support lasting positive mental health while also preventing symptoms of mental health problems are of critical importance for public health. Mental health interventions to promote well-being have the potential to not only improve women’s outcomes but also minimize the risk of negative health effect transmission to the next generation.

Health promotion strategies aim to enable optimal health and skills to cope with adversity in well subclinical populations. Therefore, it is important that efforts are made not only to deal with illness but also to develop individuals’ emotional skills that can be applied in everyday life [11]. Psychological interventions aimed at perinatal women have also been shown to be effective when delivered digitally [12]. Digital interventions—that is, computer- or web-based interventions—can be delivered offline or on the web via a computer, tablet, or smartphone. In this format, interventions can be accessed by numerous people across wide geographical regions in a cost-effective and flexible manner [13]. Web-based interventions may be particularly useful in the perinatal period, given the accessibility issues faced by this population and as many pregnant women search the internet for health information [14,15]. Furthermore, these interventions may help overcome numerous barriers that exist for women who attempt to access traditional perinatal well-being, psychological distress prevention, or treatment programs, especially challenges in navigating psychosocial care systems [13]. Widespread restrictions imposed because of the COVID-19 pandemic have generated additional barriers to accessing mental health and well-being information and services [16].

Although a recent systematic review provided preliminary evidence that web-based interventions can be a promising and advisable form of intervention during the prenatal period [13], there is a paucity of evidence on the long-term effectiveness of these programs [17]. There are many issues affecting digital mental health intervention (DMHI) implementation, such as availability issues, lack of promotion by health care providers, and lack of long-term outcome data; ultimately, program engagement is key. Low uptake of effective, evidence-based programs could diminish women’s and infants’ opportunities to enhance their well-being, limiting equitable public health benefits.

Dropout from the intervention and loss to follow-up reduces the treatment effect [18]. Although it has been argued that various strategies, including email prompts, SMS text messages, and homework are ways of helping participants develop intervention skills that can be applied, practiced, and sustained [19], it is unclear how frequently studies of DMHIs use or evaluate these strategies. Despite studies demonstrating intervention efficacy for those that remain in the study, we argue that it is just as salient to measure engagement as a benchmark of effectiveness. Web-based interventions provide tools to learn more about participant engagement and, furthermore, how it relates to retention and intervention outcomes, both in the short and the long term. This information can be used to understand the dynamics of engagement [18] and how to strengthen these characteristics in intervention development and delivery.

It is widely accepted that the full benefit of many effective treatments can be achieved only if the prescribed regime is followed reasonably closely [20]. Recent reviews [21,22] have consistently highlighted these challenges with regard to low engagement and retention rates, particularly for digital programs that often experience poor reach and uptake [13]. Sustained engagement is a complex process that has been identified as a crucial factor in intervention success [23]. However, there is a lack of systematic methodologies to assess engagement, particularly in real-world contexts. Comparing program engagement across research studies is difficult because of the wide range of strategies applied to evaluate engagement outcomes [22,24]. Accordingly, applying structured processes to assess engagement can make comparisons more meaningful.

One of the frameworks for evaluating engagement in face-to-face programs, which can be adapted to web-based programs, is the Connect, Attend, Participate, and Enact (CAPE) model [25]. The CAPE model identifies and defines 4 aspects of engagement at various stages of intervention. First, connect pertains to how many people express interest in engaging in an intervention out of those eligible. Second, attend refers to continuous presence, such as the number of intervention sessions a participant completes. Third, participate is the degree to which participants actively engage with the content of the intervention, such as completing intervention tasks and remaining in the program. The final component, enact, refers to the participant making use of intervention strategies or knowledge as part of their daily life. Although this was developed to guide face-to-face parenting program engagement research, it can be readily applied to understanding and measuring digital intervention engagement in a research context.

At a time when public resources are strained, policy makers and program administrators are looking to invest in effective, engaging prevention programs supported by scientific evidence and delivering long-term benefits. Intervention engagement must be foremost among these considerations, as this will ultimately determine the degree to which the target population takes up and benefits from the intervention when implemented in the community. Systematic reviews are an influential decision-making tool as they summarize a body of scientific research; identify implications for policy and practice [26,27]; and can be used to guide investment decisions, particularly for complex problems, such as poor intergenerational mental health.

This systematic review aimed to understand the reporting and levels of engagement in studies of web-based psychological mental health or well-being interventions administered in the perinatal period to women with subclinical mood symptoms.

Specifically, we aimed to (1) understand how studies report engagement, with engagement defined as containing the 4 steps in the CAPE model; (2) make recommendations on best practices to report engagement in DMHIs based on this; and (3) understand levels of engagement in intervention studies in this area.

To maximize the utility of this systematic review, we intended to develop practical tools for future public health use: to develop a logic model from the literature to reference the theory of change, evaluate the studies using the CAPE framework, and develop a guide for future data collection to enable consistent engagement reporting in web-based (and offline) interventions.

The review question, search strategy, inclusion criteria, and methods were registered in PROSPERO (International Prospective Register of Systematic Reviews; approval number CRD42020162283). The research question was as follows: what is known about engagement in digital mental health and well-being programs for women in the perinatal period? A systematic database search was conducted to identify papers relevant to the aims of this review. The initial search was performed by the first reviewer (JAD), using the Ovid Platform for MEDLINE, PsycINFO, EMBASE on the EBSCO Platform, Scopus, Web of Science, and Medical Subject Headings on Demand for all English-language articles published in the past 10 years (ie, from January 1, 2010, to May 29, 2020). Keywords and index terms identified as relevant in the search strategy were used and individual search criteria were developed for each database. All the database search strategies are provided in Multimedia Appendix 1. The impact of the COVID-19 pandemic delayed this publication; therefore, a subsequent rapid review was undertaken in May 2022 and performed in Google Scholar Advanced search to elicit any further publications since June 2020.

The search strategy aimed to identify studies reporting on engagement and retention in digital mental health and well-being programs for women during and after pregnancy. Clear inclusion and exclusion criteria were developed using the Population, Intervention, Comparison, Outcomes, and Study framework to guide the inclusion criteria for participants, intervention or phenomena of interest, comparators, outcomes, study design, and context (Textbox 1).

All papers that appeared eligible based on their title and abstract were retrieved for screening. The first author (JAD) reviewed the titles and abstracts of all papers, assessed eligibility, and noted any reasons for exclusion. Full-text articles were assessed for eligibility and reviewed independently by both the first author (JAD) and third author (LYG). Once the third author (LYG) had reviewed the papers, any discrepancies were resolved through team discussion. The reference lists of the included studies were examined to identify additional relevant papers.

Key article characteristics were recorded using a Microsoft Excel (version 2020) data extraction table developed for this review. These characteristics included general information about the study, such as the country and author, along with specific information about the study design, comparators, and intervention type. Coding of the study characteristics enabled us to group the studies as part of our synthesis. As our primary aim was to understand the engagement of the study population, we characterized the assessment time points, engagement measures, and reporting of attrition and adherence. Data relevant to engagement were extracted using the CAPE framework; this included variables for recruitment, retention, attrition, and follow-up time points. A framework analysis methodology [29] was used to determine which variables should be included in each step of the CAPE framework.

Logic models can help conceptualize a complex review question and specify analytic links to test the plausibility that a program works as intended [26]. Logic models typically illustrate the chain of reasoning underpinning how interventions lead to immediate (or short-term) outcomes and then to longer-term outcomes and impacts [30]. A key part of the model is detailing the mechanisms of change within the pathway and the moderating and mediating factors that may be associated with or influence outcomes. This is often referred to as the theory of change [30]. In this systematic review, the research team developed a logic model to aid the process of understanding how and when the CAPE framework could be applied to interpret the role of different engagement variables in promoting outcomes in digital perinatal mental health and well-being programs. The project team collaboratively developed the logic model, drawing on themes in the literature and the team’s collective knowledge and experience. To develop the logic model, we incorporated the types of engagement metrics found in the selected studies that could be used to assess engagement (Figure 1).

The risk of bias for studies included in this review was based on the Cochrane Collaboration’s tool for assessing the risk of bias for each category of study (ie, randomized controlled trials [RCTs] and non-RCTs), and the risk of bias was adapted for this review and classified as low, uncertain, or high based on the Cochrane risk of bias tool [31] and the primary aim of this systematic review (ie, engagement rather than efficacy). The assessment of study quality was undertaken by the first author (JAD) and reviewed by the project team. Multimedia Appendix 2 [17,32-46] provides the detailed risk of bias assessments of the included studies.

As this systematic review synthesized the results from a diverse range of interventions, we used SWiM guidelines [28] to promote transparent reporting. The SWiM items enable studies to be grouped and provide guidance on the reporting of standardized metrics used for the synthesis of findings. Specifically, we undertook the following steps:

The electronic searches generated a database of 3473 potentially eligible studies that were assessed using the review eligibility criteria. After duplicates were removed (680/3473, 19.58%), all titles and abstracts were screened for eligibility. In total, of the 3473 studies, 2795 (80.48%) records were screened, and 2654 (76.42%) were excluded based on the inclusion or exclusion criteria (Textbox 1). After the first screening, 5.31% (141/2654) of potential studies remained; the full-text articles were assessed for eligibility by the first and third authors. Of the remaining 141 studies, 125 (88.7%) were excluded on consensus by the project team; the first and third authors independently screened the papers that were verified by the team, resulting in a final selection of 16 (11.3%) studies to be included in the synthesis. The final studies were then grouped according to the study design.

The literature search and inclusion processes are detailed in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram (Figure 2). The search flow diagram indicates the papers that were selected for synthesis using the PRISMA guidelines [49]. A secondary rapid search conducted in 2022 did not yield any additional papers that met our specific inclusion criteria.

Through the analysis and reporting of each study, we recorded the types of quantitative and qualitative measures found in the selected studies that could be used to measure engagement. This enabled us to systematically construct a logic model based on our understanding of how interventions are expected to work. This was particularly pertinent for this systematic review as we did not perform a meta-analysis. As indicated, we grouped variables related to the CAPE framework; the logic model includes a range of metrics that could be systematically reported when synthesizing engagement data to visually interpret the underlying theory of change.

The logic model (Figure 1) contains 6 columns detailing the intended pathway from inputs (engagement strategies) to long-term outcomes or impacts. This approach takes a long-standing view of interventions to achieve their intended consequences. To build this model, we incorporated the types of measures undertaken in each study in this systematic review, as well as a broader range of CAPE measures found in the wider literature. The logic model indicates the point in the pathway at which the data should be collected. Mechanisms of action are factors that may facilitate engagement using a strength-based approach.

As part of this systematic review, we aimed to develop a guide for future data collection to enable consistent engagement reporting in web-based (and offline) interventions. Table 4 outlines a range of metrics that could be consistently applied in future data collection and reporting to enhance understanding of engagement and enable comparative intervention assessments.

In this systematic review, we categorized the selected studies according to study type and assessed their individual and pooled characteristics. We applied the CAPE framework [25] to all studies so that we could collectively assess and compare connection, attendance, participation, and enactment. Reporting of connection and attendance measures was fairly standardized across studies and frequently reported using a CONSORT diagram; therefore, the number of people who were eligible and expressed interest in participating, proportion of participants recruited and entered in the study, and proportion of participants who were randomized and followed up, including treatment not started and attrition rates, was clear. Approximately 75% (12/16) of studies, provided a CONSORT participant flow diagram (indicating aspects of attendance); however, the reporting categories and terminology varied between diagrams. In general, the least frequently reported domain was enactment (ie, real-world use of intervention skills), with only 38% (6/16) of studies clearly recording and reporting results such as satisfaction, usefulness, helpfulness, and perceived benefits of the skills learned in the intervention.

Some studies reported strategies to increase connection. For example, Ashford et al [13] included specific details on recruitment methods, such as social media platforms, parenthood websites, and the use of posters and flyers in clinical settings. However, other studies reported minimal details of recruitment methods (eg, Guo et al [53]). No study reported information on the background target population size; that is, the total potential pool of eligible participants. This might be a useful future metric to determine and report as an indicator of the total target population reach and the effective strategies that engage them.

As previously defined, attendance is a measure of DMHI completion through modules or similar exercises. As this was not definitively reported, for attendance, we calculated a proxy for intervention attendance as study attendance, operationally defined as intervention retention and continuous presence, both in the intervention and control arms (RCTs). Guo et al [53] had the highest participant retention and the lowest lost to follow-up participant rate in both the intervention and control groups; the intervention group showed significant improvement in depressive and anxiety behaviors. The women in this study were at a higher risk of presenting with psychological distress at baseline; although they fit our inclusion criteria, they may have had increased motivation to attend. Duffecy et al [39] undertook a user-centered design process before the pilot trial to engage women from the target population in the intervention-building process and consult on aspects such as topics, sites, and usability of potential applications. In theory, this should support attendance by reducing attrition and improving retention; however, dropout from baseline assessments to 6 weeks post partum was high (63%) [39]. We recommend that future studies report both intervention attendance and study attendance as they are distinct metrics.

A key concern in web-based interventions is the lack of participation [18]. Logs of access and use of web-based interventions can give researchers insight into people’s behavior. As Piotrowska et al [25] suggest, “The CAPE model proposes that despite the immense importance of connecting with parents and encouraging their attendance, it is active participation that has the greatest impact on parenting.” Digital interventions provide tools for learning more about participant engagement and how this relates to retention and intervention outcomes, as well as how they might be improved through the use of different ongoing engagement strategies. Crouper et al [18] quantified participant engagement using data such as dosage, exposure, or adherence. In this systematic review, few studies reported clear metrics for participation, with the exceptions of Duffecy et al [39] and Barrera et al [36]. Other suggested metrics for future research include downloading suggested resources or websites, watching suggested videos, completing quizzes and homework, or other metrics that indicate that the participant is continuously engaging with the intervention. Additional features that have potential but continue to be underexplored and underused include chatbots, games, storytelling, rewards, avatars, and personalization [42]. These features could be developed to improve participation in interventions, general app use, and studies.

Enactment is difficult to define but should be represented by measures indicating that participants put what they learned from the program into practice [24]. The limited follow-up of these studies restricted our ability to report these criteria. Only one study, Guo et al [53], followed up for any length of time, and only 31% (5/16) of studies conducted exit or follow-up interviews [13,33,35-37]. These interviews contributed more robust information on the user experience. Studies assessing skill development and use underscore the potential pathways in self-guided internet therapy, such as cognitive behavioral therapy, as mechanisms of positive clinical change [43,44]. Although these studies target clinical groups, they contribute to reinforcing the need to capture behavioral skill adaptations beyond the duration of the intervention.

Understanding the barriers to and enablers of real-world utility and practice is crucial if app developers want pragmatic uptake and efficacy of interventions. Sufficient resourcing may be a factor in longitudinal follow-up; however, to leverage the impact and cost-effectiveness of interventions, studies should factor longer-term assessments in the design process from conceptualization. Nevertheless, easier and low-cost measures of enactment are possible and suggested for future research, including questionnaires on the frequency of using skills taught during the intervention.

As part of this review, we developed a logic model to facilitate the process of gathering and integrating studies of complex interventions to better inform our interpretations of cumulative results. The logic model included synthesized data capture and engagement methods used in each study. Theoretically, logic models need moderating or mediating factors to understand how the pathway develops. In these studies, there was a common strength-based approach, such as skill development, confidence, satisfaction, and self-efficacy. Overall, the heterogeneous nature of the data collection meant that we were unable to undertake a meta-analysis; however, the range of methods and types of data collection is useful in guiding future web-based interventions targeting this population group and helping decision-makers understand the rationale for how interventions are expected to work and what enablers keep participants engaged to ultimately achieve the intended outcomes.

There is a need for a greater understanding of the individual-level, real-world factors affecting engagement in home and minimal contact practice interventions to ascertain how participants experience interventions and how this relates to their outcomes [19]. Exit and follow-up interviews can provide a deeper understanding of participants’ experiences to strengthen real-life sustained engagement in that modality. Experience of an intervention needs to be user-friendly, accessible, and positive, which should be considered in promoting material that is most effective and helpful for users to engage from the outset.

As the studies in this review were diverse in terms of study design, therapeutic intervention approach and delivery, length of follow-up, and outcome measures, we summarized the engagement data using the CAPE framework but were unable to perform a meta-analysis of the data. Attrition rates were high in many studies, and the number of participants was small, particularly in some pilot studies. We were unable to report this in terms of increasing our understanding of sustained practice as there was limited follow-up in most studies. There are inconsistent reports and terminology regarding engagement behavior. Inconsistencies in language between studies and interchangeability of terms, for example, attrition, withdrawal, dropout, and loss to follow-up, make direct comparison and systematic analysis challenging. Another potential limitation of this review is the lack of inclusion of studies in languages other than English. In addition, the protracted nature of systematic reviews means that the original search was concluded in 2020 and was affected by delays because of the COVID-19 pandemic. Since then, additional studies may have been published and not included in this review but would not necessarily affect our general conclusions or implications for using the logic model or reporting matrix.

The ability to leverage several frameworks enhanced this systematic review. The SWiM guidelines, part of the Cochrane methods, directed our synthesis and reporting. In addition, the CAPE framework provided an evidence-based approach to reporting on intervention engagement; using this framework, we were able to propose clear metrics for future reporting. It is recommended that future research provide engagement analytics to more clearly delineate between study and intervention compliance, particularly longer-term enactment or sustained practice to reflect pragmatic efficacy. The research team has a strong focus on research translation; therefore, the incorporation of a logic model provides a clear pathway for decision-makers, such as policy makers and commissioners, to interpret and guide the key constructs and evaluation metrics in future digital interventions in this field of research.

There is substantial evidence that psychological programs delivered on the web can be effective in treating and preventing mental health problems; however, the uptake of these programs can be suboptimal, and there remains a lack of evidence on how to increase engagement with evidence-based programs [45]. Poor adherence is a common feature of web-based mental health programs, which affects intervention outcomes [45] and limits real-world efficacy. Eisenstadt et al [42] discussed in their recent systematic review that adherence and retention continue to be challenges to the quality of research, with little or no information about reasons for dropouts given across studies. Further research is needed to unpack the key constructs of experience, including microlevel reporting and qualitative, phenomenological investigation via one-to-one postprogram interviews. Future reporting of DMHI using the CAPE framework could be used to ascertain the cost-benefit of an intervention; that is, if the conversion, recruitment, retention, and participation rates are high, the intervention is likely to be more cost-effective. However, this must be considered alongside the efficacy of the intervention and real-world application. The motivation for engaging in research studies is very different from real-world engagement experiences.

Advances in technology, particularly the internet, have proven to be an effective tool for building individual skills as it is inexpensive and accessible, both geographically and temporally. Despite promising results, internet interventions are still not widely disseminated or well-integrated into health services; successfully doing so will, in part, depend on engagement. As mental health apps have proliferated, choosing among them has become increasingly challenging for not only patients but also clinicians [46]. To address this, we need to understand the barriers and enablers for the delivery and sustainability of internet interventions in practice [17], as well as how we can engage not only participants but also health practitioners to support and disseminate effective interventions. This increased understanding will enable appropriate investment, optimization, and uptake of targeted well-being programs, such as those developed for perinatal women, with the ultimate aim of preventing poor mental health among women and their children.

To invest in accessible, long-term, sustainable health solutions, researchers, policy makers, and clinicians must identify optimal interventions that can be targeted to help specific risk groups or in specific contexts. Advances in technology, particularly the internet, have proven to be an effective tool for building individual skills as it is inexpensive and widely accessible. Pregnancy and the postnatal period can be times of increased psychological distress; therefore, it is an optimal time to intervene with strength-based tools to build affirmative self-efficacy. Although several studies in this field demonstrate efficacy, few robustly explore the construct of engagement, and in particular, there is limited evidence of the long-term enactment of the strategies learned. Our results indicate a disparity in the reporting of short- and long-term participant engagement behaviors, and we recommend the adoption of standardized metrics for reporting DMHI engagement in both research and real-world settings. This systematic review provides a framework for understanding the pathways for enhancing the mental well-being of mothers and their infants. With the world experiencing an endemic escalation in poor mental health across the life course, both in low- and high-income countries [55], it is imperative that we create practical, evidence-based, cost-effective, and scalable solutions to protect current and future generations.