There has been a proliferation in the number of smartphone apps being developed, both commercially and in academic research programs, which aim to improve mental health and well-being. Recent estimates suggest that anywhere from 10,000 [1] to 22,750 mental health apps exist [2]. Although many of these apps can be accessed directly by individuals in the commercial app marketplace as self-care tools, they are also playing an increasing role in clinical services, supplementing or enhancing traditional interventions [3]. The rapid expansion in the research and development of mental health and well-being apps highlights how much interest and potential there is thought to be in the mobile health arena.

Most common mental health disorders, including depression and anxiety, have their onset during adolescence and, if not successfully resolved, can lead to negative impacts well into adulthood [4,5]. Given the increasing number of young people (between the ages of 15 and 25 years) using digital technologies, smartphone-based interventions provide a scalable solution to support this group to manage their mental health and well-being [6]. Apps have the potential to address some of the accessibility issues in service provision for young people’s mental health, especially for underserved populations. Emerging evidence suggests that some apps may produce significant symptom improvement across multiple outcomes compared with waitlist or control conditions [7-9]. Despite this promise, empirical research often fails to translate into meaningful and sustained implementation in “real-world” settings [10,11]. This can be attributed, in part, to the complex and lengthy process of implementing and maintaining evidence-based approaches in practice, as well as the commercial and regulatory complexities of scaling up mobile technologies in health services [12,13]. Besides innovation and efficacy, other factors, including user engagement, usability, acceptability, accessibility, and low cost, are key prerequisites for adoption, scalability, and uptake [14-17].

Given these multifaceted challenges, it is important to identify what facilitates and inhibits the implementation of mobile mental health interventions [10]. Currently, our understanding of how mental health apps are implemented in real-world settings is limited in several ways. Foremost, implementation processes and outcomes from research trials are seldom recorded or reported, and implementation efforts often lack a solid theoretical or model-based approach, making it difficult to understand and explain how and why implementation succeeds or fails [18,19]. In the context of mobile mental health apps, successful implementation can be measured by the extent to which the intervention has been embedded into service provision, the number of app users, the frequency of app use, app engagement, and evidence of sustained use following the end of a research trial [20,21]. Assessing implementation outcomes using a conceptually grounded framework allows for a systematic assessment of outcomes while also supporting the rigor and reproducibility of implementation research and providing building blocks for the implementation of future interventions.

Existing scoping and systematic reviews have focused on reviewing and critically appraising the methodological rigor and quality of implementation effectiveness studies, reporting implementation outcomes as their primary outcomes [22,23]. To the best of our knowledge, no review has taken a systematic approach to assessing the successful implementation and sustainment of all evidence-based mental health apps for young people. In this review, we assessed the factors influencing implementation success according to a set of implementation outcome criteria based on a modified version of the implementation framework by Proctor et al [24]. In total, 10 implementation variables were examined, of which 8 were from the Proctor model: acceptability (perceived usefulness and satisfaction with a technology), appropriateness (fitness for purpose), feasibility (extent to which a technology was successfully used), fidelity (implementation as intended), cost (financial impact of technology implementation), adoption (technology uptake and use), penetration (spread or reach of the technology), and sustainability (sustained uptake by users or maintenance or integration of a technology within a health care service). Two additional relevant outcomes were added: coproduction (user involvement in intervention development and evaluation) and engagement (adherence and dropout) [25,26].

Although conceptually distinct constructs, the implementation variables listed above are dynamically interrelated and sequentially contingent on one another [24,27]. For an app to be engaging, widely adopted, and well sustained, it must first be acceptable, appropriate, and feasible. It may be that an app-based intervention is deemed highly relevant and applicable to young people’s needs (high appropriateness) but may be costly to download and time intensive (low feasibility). Similarly, an intervention may be considered by a mental health service as a good fit to address young people’s needs (high appropriateness); nevertheless, the service user may be reluctant to use it if they dislike a certain feature of the intervention (low acceptability). Given the potential benefits of smartphone apps in supporting the mental health and well-being of young people, it is critical that researchers and app developers place greater emphasis on enhancing the engagement, implementation, and scalability of efficacious interventions in local contexts or specific populations.

The aim of this review was to determine how successfully evidence-based mobile apps, which aim to promote well-being and mental health outcomes in young people, are adopted, scaled up, and sustained in real-world settings. The research questions of interest were as follows:

An information specialist (EH) performed an electronic search of the following databases from January 1, 2011, to the search date on February 2, 2021: Ovid Embase, Ovid MEDLINE, Ovid PsycINFO, Cochrane Database of Systematic Reviews, and Cochrane Central Register of Controlled Trials. The search strategies used text words and relevant indexing to capture the concepts of studies on the effectiveness or trials of mental health apps for young people. The search strategy was guided by similar reviews exploring digital mental health interventions for young people [25], and the terms for apps were derived from Cochrane reviews [28,29]. The full search strategies are available in Multimedia Appendix 1. Duplicates were removed following the method described by Falconer [30], and records were then screened by titles and abstracts to complete the process manually. The reference lists of included studies and relevant systematic reviews were assessed for additional relevant studies. All references were exported to Endnote X9 (Clarivate) and then to the systematic review software Rayyan [31].

The focus of this review was the implementation of app-based interventions that aim to promote mental health and well-being, prevent mental health problems, and treat existing mental health problems in young people. Screened articles were included if (1) the study targeted young people with a mean age of 15 to 25 years, with or without a formal mental health or physical health diagnosis (eg, targeting anxiety in adolescents with diabetes); (2) the intervention was an efficacious “native” mobile app (ie, not on a web browser), whose primary aim was to promote well-being, prevent mental health problems, or treat existing mental health problems; (3) the primary outcome was a measure of mental health or well-being, including change in anxiety and depressive symptoms, diagnosis, problem severity, problem improvement, recovery, remission, or more general change in mental health or well-being across at least 2 time points (eg, baseline and after the intervention or follow-up); and (4) the intervention was efficacious, that is, it had beneficial mental health or well-being outcomes compared with any other type of digital intervention, usual care (eg, psychotherapy), waitlist control group, or no-intervention control group. Both randomized and nonrandomized studies were considered for inclusion.

Articles were excluded if (1) the mean age of participants was not 15 to 25 years; (2) the intervention was not a mobile app, such as other digital interventions, including therapy delivered by phone, SMS text message, video platforms, or PC (eg, computer-based cognitive behavioral therapy); (3) the apps used were not efficacious (ie, there was no significant improvement in mental health or well-being compared with a control group); (4) the apps used did not include an intervention component (ie, primarily focused on diagnosis or assessment); (5) the studies did not report mental health outcomes or the primary outcome was physical (eg, blood sugar levels or exercise); and (6) the studies did not have a control group. Gray literature was not included in the search.

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [32], the flowchart presented in Figure 1 provides step-by-step details of the study selection procedure. The search strategy identified 18,660 citations after the removal of duplicates. Of these 18,660 studies, 1634 (8.75%) were considered potentially relevant based on their titles and abstracts. Three members of the review team (HB, LAN, and JD) screened the titles and abstracts against the inclusion criteria. The remaining full texts were screened by 6 members of the review team (HB, LAN, JD, SL, BM, and LN). At this stage, 20% of the texts were screened by at least 2 reviewers independently to ensure interrater reliability (intraclass correlation coefficient range 0.88-0.96). Any disagreements between the 2 reviewers were resolved through discussion with the wider review team.

Data were extracted by 1 reviewer (HB or LAN) and reviewed for accuracy and completeness by another. After verifying all the extracted data, discrepancies were resolved through discussion or adjudication by another party (MF). Extracted data included information on study characteristics (eg, authors, publication year, country, study design, and study population), intervention characteristics (eg, characteristics of the technology, app name, therapeutic modality, and intervention outcomes), and implementation constructs (eg, implementation objectives and implementation results). Textbox 1 provides a description of the implementation outcome criteria.

The mixed methods appraisal tool (version 2018) was used to assess the methodological quality of the included studies [33]. It was developed by combining the core relevant methodological criteria found in different well-known and widely used qualitative and quantitative critical appraisal tools. It consists of 2 screening questions applicable to all types of study design and a further 5 questions applicable to specific study designs. Responses were rated on a categorical scale as “no,” “unclear,” or “yes” to any of the methodological quality criteria. Quality assessments were made by 1 of 3 reviewers (SL, BM, and LN). We did not exclude any studies based on quality assessment scores.

The extracted data were collated and summarized to produce a narrative summary of the study, sample, and intervention characteristics. To determine the proportion of apps that were sustained or adopted after development, we contacted the corresponding author of the included articles to complete a brief survey about the development and implementation of the app described in their study (Multimedia Appendix 2). If they did not respond, the first or lead author was then contacted. In addition, we searched to check the availability and discoverability of the app in the Apple App Store (iPhone or Mac) and Google Play Store. A codebook approach was used to code and synthesize implementation data from all available sources according to the 10 implementation outcome categories [34].

The characteristics of the included studies are presented in Multimedia Appendix 3 [35-68]. Of the 34 studies, 11 (32%) studies were conducted in the United States; 6 (18%) in the United Kingdom; 3 (9%) in Australia; 2 (6%) each in Italy, Japan, New Zealand, and Spain; and 1 (3%) each in Canada, Iceland, Iran, Israel, South Korea, and Sweden. Most studies (31/34, 91%) were published between 2018 and 2021. Regarding the evaluated sample populations, most studies (24/34, 71%) primarily recruited university students, followed by clinical samples (5/34, 15%), general population samples (5/34, 15%), school students (3/34, 9%), primary care patients (1/34, 3%), and those attending youth organizations (1/34, 3%). In terms of study design, 74% (25/34) of studies were randomized controlled trials (RCTs), 15% (5/34) were pilot RCTs, 6% (2/34) were quasi-experimental, and 6% (2/34) were feasibility trials. Notably, 29% (10/34) of the included studies evaluated the effectiveness of a commercially available app that was not developed by the evaluation study team.

Multimedia Appendix 4 [35-68] outlines the format and delivery of interventions assessed in the included studies. Of the 34 studies, 15 (44%) studies aimed to treat mental health problems or reduce symptoms, 14 (41%) aimed to promote well-being or mental health, and 5 (15%) aimed to prevent the onset of mental health problems. Most apps studied (29/34, 85%) were stand-alone, and the remainder (5/34, 15%) accompanied other therapeutic interventions.

The measurement of successful implementation, such as sustained use following the end of the research trial either commercially (eg, discoverable in app stores) or otherwise (including available to young people via schools or mental health services) varied based on unique features of the app itself, its recipients, and its context. To ascertain the proportion of apps that were sustained after development, we contacted the authors of the included articles to request additional information about the implementation of the app reported in their study. We did not contact the authors of the 18% (6/34) of studies testing existing apps that we knew were commercially available and discoverable to the public at the point of the review (eg, Headspace and Calm); that is, being successfully sustained. Of those contacted (28/34, 82%), we collected 23 survey responses (Multimedia Appendix 5 [35-43,48,50,51,54,55,57-61,63,65-67]), and 5 authors did not respond. However, on reviewing the survey responses, 17% (4/23) were evaluating other apps already available. In the absence of survey data for 5 studies, we checked if they were commercially available; 2 (40%) were discoverable on either the Apple App Store (iPhone or Mac) or Google Play Store (Google Inc) [45,62], and 3 (60%) were not available [52,53,64].

In summary, 10 articles evaluated the effectiveness of an existing app that was available for use at the time of the evaluation study [38,44,46,47,49,54,56,59,60,68]. Of the 10 articles, 8 (80%) evaluated an app still available on the market: Headspace [44,46,47,68], Calm [49], Pacifica/Sanvello [38], Smiling Mind [47], Stop Breathe Think [56], and Thrive [59]. The 2 major smartphone app markets (ie, App Store and Google Play) publicly list app ratings out of 5 on their store pages. Google Play also provides download count estimates. The consumer app ratings on a 5-point scale, from the App Store and Google Play, respectively, are as follows: Headspace (rating 4.9 and 4.25) and Calm (rating 4.8 and 4.25) were the most popular (>10 million downloads), followed by Sanvello (rating 4.8 and 4.5), Stop Breathe Think (rating 4.8 and 4.5), Smiling Mind (rating 4.5 and 3.75; >1 million downloads), and Thrive (rating 5.0 and 3.5; >50,000 downloads). Finally, 2 studies evaluated consumer apps that are no longer available: DeStressify [54] and Lantern [60]; therefore, their store statistics are not reported.

Of the 24 studies reporting on a newly developed app, 43% (9/21) are currently available, commercially or otherwise (eg, in mental health services), and 57% (12/21) are no longer available. Most respondents reported that it took several years to develop and test the app reported in their articles, ranging from 6 months to 6 years.

Methodological quality varied across the included studies (Multimedia Appendix 7 [35-68]). Most studies (32/34, 94%) were judged as having possible limitations in at least 1 criterion. Most studies (29/34, 85%) clearly described the randomization of the study participants or the process for recruiting a representative sample. Most randomized trials reported complete outcome data (30/32, 94%) and described samples that were comparable at baseline (27/32, 84%). However, few studies described the process used to blind the outcome assessor to the intervention group (10/32, 31%). Only half of the studies (16/32, 50%) reported acceptable adherence rates to the intervention. Regarding the 2 nonrandomized trials, both studies used appropriate outcome measures. Of the 2 studies, 1 (50%) was judged as not clearly accounting for confounders in the study design and analysis. Both studies were judged as adhering to the intervention protocol [38,53].

The primary aim of this systematic review was to determine the proportion of evidence-based mental health and well-being apps that have been successfully adopted and sustained in “real-world” settings. In total, 29% (10/34) of studies were identified that evaluated the effectiveness of 8 existing, commercially available mental health apps. The remaining 71% (24/34) of included studies evaluated 21 newly developed apps, of which 43% (9/21) are currently available, commercially or otherwise (eg, in mental health services), and 57% (12/21) were no longer available at the time of enquiry. Therefore, these results not only indicate a 43% implementation success rate of new apps but also provide some information on how existing efficacious commercial apps are promoted and sustained in the field.

Broadly synthesized using 10 dimensions of implementation, our review suggests that measures of adoption, acceptability, appropriateness, and feasibility are more frequently reported than indicators of cost, fidelity, sustainability, and penetration. Implementation outcomes were unavailable for many of the studies, precluding direct comparisons between those apps that were implemented and those that were not across constructs. To partially address our second research question, most of the apps that had been implemented confirmed a degree of adoption, acceptability, appropriateness, feasibility, and engagement. These determinants are relevant to a range of interventions and would benefit from broad systematic incorporation into the development of a smartphone app. Other important factors identified included the coproduction of interventions with young people and the need to embed apps within local settings, such as schools, universities, and mental health services, rather than relying on commercial strategies. Although assessing implementation outcomes using an integrated framework allows for a more systematic assessment of outcomes, the review highlights a lack of measurement precision around implementation constructs in that an array of overlapping terms, such as acceptability and usability, are often used interchangeably, highlighting the need for greater consensus on how to measure and report implementation determinants. There is also a need to elucidate the relationship between different implementation constructs (eg, acceptability and engagement) and intervention effectiveness (eg, clinical change) [22]. Modeling these relationships was not possible in the current review because of data quality issues. Further empirical research is needed to model the interrelationships between implementation variables to better understand the nature of their connections and their impact on implementation success.

Confirming acceptability and engagement in the early phases of intervention development, before implementation commences, is an important step in ensuring that the intervention has potential longevity and may be important for saving time and money. As evidence suggests that engagement is the highest in the first days or weeks of downloading an app, it might be more appropriate for apps to be developed with this anticipated behavior in mind [72].

Research has identified that intervention- and person-specific factors that influence engagement with mobile mental health interventions should be considered [25]. For example, getting individuals to use an app just once is an important step in retention. It is likely that an initial hook is needed to catch the attention of young people to encourage them to identify and download the app. Thereafter, rewards and gamification are reported by young people as motivating factors for ongoing engagement and retention [25]. Similarly, usability has been reported as important for promoting engagement. Acceptability and engagement are critical elements of mental health apps that support users [73]. However, it is recognized that operationalizing meaningful app engagement is not straightforward, as many downloaded apps are never used and more work is needed to define sustained engagement, what leads to it, and how to create products that achieve it [73,74]. Further research may explore characteristics facilitating initial use, continued use, and the implications of changing the treatment structure during an intervention (eg, Do individuals find it motivating or overwhelming?).

The importance of identifying mental health interventions that are efficacious needs to progress in concert with addressing implementation requirements from the outset of app development. This can be a complicated focus for researchers and funding bodies, as in the absence of efficacy, it might seem premature to address dissemination issues, but given the high redundancy of scientific studies—less than half of the efficacious apps were actually identified as being in sustained use—these questions must be paramount at the outset of the research cycle.

In addition to the markers of successful implementation, there are several extant barriers, including high cost, time, funding constraints, and lengthy research processes. The process of applying for funding, app development, data collection, data analysis, and app release is lengthy and expensive, taking up to 6 years [39]. The sustainability and penetration of apps is contingent on projects being funded or grants being successful, and even when funding is secured, it is often time limited [42,67,75]. The way in which funding streams are set up means that it is harder to secure funding for the follow-on implementation research than for the initial evaluation project.

Other challenges include the continuous, rapidly evolving development of technology, which results in the need for continued code updates and upgrades to allow for compatibility with smartphone operating systems [42,61]. Additional noted barriers include the rapidly changing nature of many commercial providers supplying the app technology, sometimes with significant staff movement, changes to the focus of work depending on commercial drivers, and short life spans of some tech companies [60]. Given that only a small proportion of existing, commercially based apps are being well sustained, rather than developing de novo apps, it might be that evaluating more successful apps and adapting them to specific contexts can propel implementation in the field. However, it is recognized that the commercial app marketplace is highly competitive, and research has identified that, based on our estimates of monthly active users, mindfulness and meditation apps appear to be the most popular: Headspace and Calm account for 13.4 million users, Replika and Wysa account for 1.5 million users, and Reflectly and Daylio account for 840,000 users [76]. In total, these 6 apps monopolize the marketplace, many of which have been made freely available to young people and extensively marketed; hence, they penetrate the market and account for 83% of the monthly active users of mental health and wellness apps [76]. As noted by others, currently, the mental health and well-being apps that have been rigorously evaluated struggle to attract users, and apps with many users are rarely evaluated [76,77].

There are several possible explanations for the success of these commercial apps. Fish and Saul [44] reported that Headspace, one of the most popular consumer apps in the market, applies gamification techniques, such as accomplishment, empowerment, social influence, and ownership to improve engagement and motivate individuals to enjoy meditations and find them rewarding. Several engagement design features in Headspace, such as user tracking, reminder function, and push notifications, can improve engagement, these features were not actively part of any of the evaluated studies and were left for use at the discretion of the individuals [46]. However, Huberty et al [49] report that these features may be critical for engagement and acceptability.

An interesting finding of this review is that there are certain groups that are less likely to access mental health and well-being apps. It is noteworthy that most studies included in this review recruited university students (24/34, 71%). There was a marked absence of youth samples from underserved or marginalized populations, including but not limited to migrants, asylum seekers and refugees, those experiencing homelessness, and those from socioeconomically deprived backgrounds. These potentially high-risk groups are typically underrepresented in research, face access and engagement barriers when navigating health care systems, and experience digital exclusion [78]. The studies included in this systematic review had not targeted these populations, and research exploring the acceptability, appropriateness, and feasibility of app-based interventions is lacking.

Although this review was rigorous, carefully executed, and used a robust methodological approach, it was not without limitations. Foremost, although the review team attempted to identify and include as many articles as possible, some articles may have been missed because of the inconsistencies in how implementation outcomes are recorded and reported. It was also difficult to ensure that all apps for this age group were identified because those aged between 15 and 25 years are harder to differentiate in adolescent and adult studies, meaning we might have missed some relevant studies. Although the focus of this review was young people aged between 15 and 25 years, most of the included studies recruited university students, and most of the study samples had a mean age of >20 years. This finding suggests that there may be a lack of effective mental health apps targeted at mid to late teenagers. This may be a result of the ethical constraints of recruiting young people aged <18 years (16 years in the United Kingdom) to research studies, as it is easier to recruit those who are deemed able to independently consent to trial participation. Additional research is needed to determine whether mental health apps have been developed for this group; however, upon evaluation, they were deemed ineffective. Given the considerable heterogeneity in the social, emotional, and cognitive development and maturity of young people in this age range, research is needed to understand the potential differential impacts and utility of apps throughout adolescence and young adulthood.

In addition, the review included several apps that were delivered as an adjunct to in-person therapies. It is recognized that the reported successful implementation of these apps may in fact be a by-product of the success of the primary therapeutic intervention rather than the app itself. Research comparing face-to-face therapies with and without additional app support provides evidence to support the superiority of adjunctive interventions compared with standard intervention-only conditions [79]. However, additional research is needed to enable us to determine the role of apps in the successful implementation and sustainability of blended intervention programs.

Unpublished data were not included in the search, which may have affected the results of this review. Nevertheless, this approach was also seen as a further strength by ensuring that only peer-reviewed interventions were included. Given that the focus of our review was on apps that had been found to be efficacious, the publication bias effect was minimal. However, it is possible that companies that have developed the apps possess valuable data pertaining to some aspects of implementation. Commercial companies are likely to have collected a wealth of relevant and informative data on user engagement and app use over several years. However, owing to competing commercial interests and a lack of regulation in the mobile mental health app arena, these data are not publicly available or made openly accessible to researchers [80]. Nonetheless, an attempt was made to obtain implementation data when it was not included in the articles by contacting the study authors, most of whom responded.

Finally, more than two-thirds of the apps were not independently evaluated and, therefore, an important consideration is that “developer bias” may have impacted the analysis and subsequent findings of those trials. More independent and robust evaluation of apps is needed, and findings must be shared in a manner that is accessible to the scientific community as well as the users of the app. This will require time and resources but needs to become integral to the development process to mitigate potential bias in evaluations moving forward [81], be that within purely commercial or research contexts or for commercial-research collaborations.

Despite the significant amount of funding that has been directed toward the development of mobile mental health interventions, few have published evidence-based data to support their use in real-world settings, and even fewer have been successfully transitioned into sustainable mental health interventions. Although it had been thought that smartphone apps held the potential to address many of the current issues facing service provision in youth mental health by improving the scalability and affordability of evidence-based mental health interventions for young people and addressing health disparities by providing wider access to underserved populations, more work is needed to improve key implementation drivers, such as uptake and adoption. Innovative and targeted funding mechanisms that are quick, responsive, and encouraging of broad stakeholder and industry partnerships, where data are openly shared, are essential to ensure mental health and well-being app development, evaluation, implementation, and sustainability proceeds in a direction that will enable evidence-based interventions to be made available quickly to young people who may benefit from them.