Anxiety disorders among young people are a significant public health concern, affecting nearly 7% of young people aged 4 to 17 years [1]. In recent years, multiple national and global crises have occurred, including ongoing cost-of-living pressures, climate and weather-related events, and the COVID-19 pandemic, all of which have been associated with increases in mental health problems [2]. One contemporary Australian multiwave cohort study demonstrated that 58% of adolescents reported clinically significant anxiety symptoms at some point during adolescence [3]. Thus, there is an urgent need for scalable solutions that can reach the many young people impacted by anxiety. Open access digital mental health platforms (DMHPs) have been highlighted as a promising way to increase access to evidence-based interventions and represent an approach that is well accepted both by adults [4] and young people [5]. Based on a representative large-scale Australian survey, a higher proportion of adolescents (20.1%) use online mental health programs and resources compared to either conventional health services (18.3%) or school-based supports (2.4%) [6]. DMHPs encompass websites, applications, and other digital systems that deliver guided and unguided mental health assessments and evidence-based interventions, such as cognitive behavioral therapy (CBT) programs. Unlike traditional mental health services that deliver mental health interventions in person (either face-to-face or via telephone), and crisis support services (eg, Lifeline and Kids Helpline) that provide immediate in-person support via telephone or synchronous messaging, DMHPs deliver intervention content digitally, often in self-directed formats via modules and programs. This delivery model, in which the therapeutic content is embedded in a program, can serve as a general avenue for enhanced accessibility to evidence-based interventions; however, the widespread implementation of DMHPs in real-world contexts requires further examination.

Several nationwide DMHPs targeting adults have been implemented in Australia, including THIS WAY UP [7] and MindSpot [4]. MindSpot Clinic is an e-therapy clinic for adults that attracted 121,652 new program users in the first 7 years of clinic operation [4]. Importantly, MindSpot offers assessments and interventions, with an average of 67% of users using the site primarily for assessment and information purposes [4]. Users of this freely available digital program have reported high satisfaction and demonstrated significant improvements in psychological symptoms (eg, anxiety and depression), with large effect sizes [4]. THIS WAY UP offers both guided and unguided programs, with users showing substantial improvements in psychological distress, anxiety, and depression symptoms following online treatment [7,8].

With respect to children, BRAVE-ONLINE is an accessible, evidence-based, and effective DMHP for the treatment of anxiety in Australian children and adolescents. Several randomized controlled trials have demonstrated the efficacy of various BRAVE-ONLINE models of care, including therapist-assisted [9-11], blended care [12], and stepped-care [11,13]. The self-help version of BRAVE-ONLINE (BRAVE Self-Help) was developed in 2014 to deliver a public health model of care aimed at increasing access to evidence-based treatment for child and adolescent anxiety and has been validated in 2 open effectiveness trials [5,14]. In these 2 studies, significant reductions in anxiety were observed for participants who presented with initially elevated levels of anxiety and who engaged with the program [5,14]. Further, March et al [14] demonstrated that significant improvements were made by users presenting with moderate to severe levels of anxiety, although the authors suggested that those with severe anxiety might need additional ongoing support given their average final anxiety score remained in the “elevated” range. Thus, the BRAVE Self-Help program provides an innovative care model with the potential to significantly reduce the national impact of childhood anxiety problems.

Although DMHPs demonstrate promise in handling the ever-increasing demand for mental health services, there is an absence of literature grounded in implementation science frameworks that details the implementation strategies, implementation and effectiveness outcomes, and real-world experience of disseminating them. Implementation science frameworks assist researchers in effectively translating evidence-based interventions into practice and testing them in real-world conditions [15-17]. These include taxonomies that aim to provide recommendations for effective implementation strategies that can be used to enhance adoption, implementation, sustainment, and scale-up of an evidence-based practice, such as Powell et al [18], and taxonomies for assessing implementation outcomes. For example, Proctor et al [19] developed the implementation outcomes framework (IOF), which presents a taxonomy of implementation outcomes to assess implementation success, including acceptability, adoption, appropriateness, feasibility, fidelity, penetration, and sustainability. More recently, efforts have been made to contextualize implementation frameworks and metrics to digital interventions, particularly those that involve minimal provider mediation and greater data collection at the consumer level [20]. Further, the potential of hybrid effectiveness-implementation designs that combine elements of effectiveness and implementation research has highlighted as a strategy for enhancing the translation of research findings into routine adoption [21]. As DMHPs become more embedded in community and routine care settings, there is an urgent need to examine both effectiveness and implementation outcomes to ensure successful translation.

Previous studies have examined the feasibility and acceptability of the BRAVE Self-Help program in subsamples of young people using the program. This study aimed to describe the real-world implementation of BRAVE Self-Help via an open effectiveness-implementation trial delivered in a community setting and to comprehensively examine its outcomes over the first 10 years of operation from 2015 to 2024. This study goes beyond prior evaluations of BRAVE Self-Help (focused on establishing initial feasibility and symptom trajectories in subsamples of participants), extends the time frame to 10 years, and examines the implementation outcomes in detail, within a clear implementation science framework. To provide context for the implementation and effectiveness outcomes of the program, the implementation of BRAVE Self-Help was retrospectively documented, including implementation strategies during its establishment and sustainability stages. The extent to which these strategies align with those identified by Powell et al [18] as being important in effective implementation was also examined as a supplementary aim of the study. A detailed description of these implementation activities is provided in Multimedia Appendix 1.

The primary aim of this study was to report on the overall implementation and effectiveness outcomes of BRAVE Self-Help from 2015 to 2024. In doing so, this study drew from Proctor et al’s [19] IOF and Hermes et al’s [20] taxonomy of implementation outcomes for digital interventions with minimal provider mediation. By examining the use and outcomes of the platform over the first 10 years of the program’s implementation, this study aimed to illustrate how a DMHP such as BRAVE Self-Help can provide a viable service delivery approach capable of managing surges in mental health service demands over time. This study will also serve to inform policymakers and health care providers regarding the characteristics of young people accessing DMHPs and will inform the future design and delivery of digital mental health services going forward.

This paper reports on an open effectiveness-implementation study intended to evaluate the implementation and effectiveness of the BRAVE Self-Help program when delivered in the community over a 10-year period as an open-access anxiety program for children and adolescents.

The study received ethical clearance from the 3 main institutions involved in the development and distribution of BRAVE Self-Help – the University of Queensland (2013001397), Griffith University (2020/581), and the University of Southern Queensland (H20REA298).

Participants included children and adolescents who registered for the publicly available BRAVE Self-Help program for anxiety in Australia. Children and adolescents could self-refer to the program, or be referred by parents, schools, general practitioners, and mental health professionals. The platform is open to families using a computer with an Australian IP address, and can be accessed at any time, from any location. Upon visiting the website, potential users are required to provide informed consent prior to accessing the program. For users aged <16 years, parental consent is also required. Participation in the program is entirely voluntary, and users are informed that they may cease using the program at any time without consequence. Elevated levels of anxiety are not a requirement to engage with the platform; however, for the effectiveness analysis, only users who presented with elevated levels of anxiety were included. Elevated anxiety was classified as users with a score on the Children’s Anxiety Scale 8-item (CAS-8) ≥84^th percentile (≥T score 60; CAS-8 score ≥10 for boys, ≥11 for other gender, and ≥12 for girls). Access to the program remains open during and after completion of the program, unless the user requests closure of the account.

Participants comprised all children and adolescents registered for the BRAVE Self-Help program from January 1, 2015, to December 31, 2024. Although the program officially launched in mid-2014, the first 6 months constituted a “soft launch” period during which changes were made to processes, the establishment of implementation and refinement of implementation strategies, as well as changes to the data collected. A consistent dataset was in place from January 2015, along with refined implementation strategies for full launch. Thus, January 2015 is considered the active commencement of the program.

BRAVE Self-Help is an internet-based cognitive behavioral therapy (iCBT) program for child and adolescent anxiety. It includes separate programs for children (7‐12 years) and adolescents (13‐17 years). Child- and adolescent-focused programs include 10 interactive iCBT sessions presented in an engaging manner designed to facilitate motivation and learning. Sessions are completed sequentially, unlocking after each session is completed. Session activities include content delivery (reading and video), skill rehearsal activities, quizzes, and games to consolidate knowledge, and homework tasks to facilitate practice and learning of skills outside the session. Age-appropriate language and graphics are used for each of the programs, as well as age-specific content and examples. Child and adolescent sessions include evidence-based anxiety management strategies such as recognition of physiological anxiety symptoms, relaxation, coping self-talk, cognitive restructuring, and graded exposure. The program tracks the young person’s anxiety symptoms each session and includes clinical tools to assist with CBT exercises (eg, an online tool to develop and implement an exposure hierarchy; a “relaxation room” with resources targeting emotion regulation). For examples of program pages, see Figure 1. For a detailed description of the online program, see previous studies [5,22].

Anxiety severity was measured prior to beginning the program at registration, at the commencement of session 1 (if the program was not commenced within 7 days of registration), and at the commencement of each subsequent BRAVE-ONLINE session using the CAS-8 [24]. The CAS-8 includes 8 items that require participants to rate their level of anxiety on a 4-point response scale ranging from 0 (never) to 3 (always). Total scores may range from 0 to 24, with higher total scores indicative of greater levels of anxiety. Example items include “I worry what other people think of me” and “I worry that something bad will happen to me.”

For the purposes of this study, users were categorized into “no elevated anxiety,” “elevated anxiety,” or “clinical anxiety” based on population-level and gender-standardized norms produced via large-scale, school-based prevention program trials [24]. Scores falling <84th percentile (<T score 60; CAS-8 score ≤9 for boys and ≤11 for girls) were categorized as “no elevated anxiety,” scores falling ≥84th percentile (≥T score 60; CAS-8 score ≥10 for boys, ≥11 for other gender, and ≥12 for girls) were categorized as “elevated anxiety,” and scores ≥94th percentile (T score ≥65; CAS-8 score ≥13 for boys, ≥15 for other gender, and ≥16 for girls) were categorized as “clinical anxiety.” The CAS-8 has demonstrated good psychometric properties (Cronbach α=0.89) [24], and for this study, the average internal consistency of the CAS-8 across timepoints was 0.88.

Program satisfaction is measured via a 5-item scale that gauged the user’s perception of program acceptability at 3 timepoints (after sessions 3, 6, and 9) [5]. For the purposes of this study, the final satisfaction survey the user completed (either after 3, 6, or 9 sessions) was used. Responses are rated on a 5-point scale (1=“definitely not” to 5=“definitely yes”), with total scores ranging from 5 to 25. Higher scores indicate higher levels of satisfaction. Question items include whether users would tell a friend about the program, how helpful they found the program, and how happy they were with the program [5].

Data relating to use of the program were tracked automatically throughout program participation. The number of new program registrations was captured through total and monthly registrations. The number of users completing the baseline assessment (and therefore able to commence the program) was also calculated. Program completion was operationalized as the number of sessions completed by users (ranging from 0 to 10), with the mean score calculated. The proportion of users completing each session was also calculated.

To address the primary aim of this study, the implementation and effectiveness outcomes presented in Table 1 (based on Proctor et al’s [19] IOF and Hermes et al’s [20] taxonomy) were analyzed across 10 years. Deidentified data extracted from the BRAVE Self-Help program were visualized using Microsoft Excel Worksheet and Canva and analyzed using SPSS software (version 28.0; IBM Corp). To describe the overall trends over the 10-year period, the number of new registrations per month and year were graphically represented according to school calendar periods, natural disaster events that occurred during each annual period, and the COVID-19 pandemic. School-term dates were compared Australia-wide and holiday periods were mapped for each year. Community- and nationwide natural disaster events were defined as events directly threatening or damaging property or life, with this damage being widespread, and for continued periods, including bushfires, floods, cyclones, supercell storms, heatwaves, and pandemics [25-28]. A negative binomial regression model using a log link function was used to investigate the association of weather events with total monthly registrations while controlling for school holidays, the COVID-19 pandemic, seasonal, and yearly trends.

In terms of participant characteristics, chi-square analyses were conducted to compare baseline characteristics between program completers (10 sessions) and noncompleters (<10 sessions) and between users who did not start the program and those who completed 1 or more sessions. Additional analyses were conducted to examine the associations between engagement level and baseline characteristics. Finally, a one-way analysis of covariance (ANCOVA) was used to determine differences in final satisfaction scores, according to baseline characteristics, while controlling for the number of sessions completed by the participant.

Program effectiveness was examined using 2 methods. First, changes in anxiety symptom (CAS-8) scores were examined. Paired-samples 2-tailed t tests were used to determine changes in anxiety symptoms from baseline to the user’s final recorded anxiety rating, irrespective of the number of completed sessions. That is, if the user completed all 10 sessions, their final anxiety score would be at session 10. If the user completed only 4 sessions, their final anxiety score would be at session 4. In this way, the change in anxiety captures the change made by the person according to the amount of treatment they received. This analysis was conducted for all users including those with and without elevated anxiety when registering.

Given that few participants provided data at all assessment points, we also analyzed the full sample through mixed modeling. The sample was split by elevated or nonelevated baseline anxiety, using time (baseline CAS-8 and final CAS-8 score) as a fixed effect and a random intercept for participants, to confirm the findings of the completer analysis for both primary and secondary analyses, especially among those with elevated anxiety. A sensitivity analysis repeating the mixed models with those participants who completed all 10 sessions of the program was performed to assess the robustness of the analysis to missing data. Furthermore, to investigate changes in anxiety scores at specific points in the program (baseline to session 4, baseline to session 7, and baseline to session 10), separate repeated measures ANOVA tests were conducted for those who had elevated anxiety at program registration and had completed the sessions (eg, sessions 1‐3) and CAS-8 measures up to that point.

Second, the proportion of users making clinically meaningful change on the CAS-8 from their first to last session was calculated for participants shifting from elevated or clinical categories at baseline to nonelevated anxiety or elevated anxiety at their last session. The degree of clinically meaningful change was also examined for users with “normal” anxiety classification at baseline; specifically, to identify the proportion of participants who showed meaningful deterioration into an elevated or clinical category, from their first to last session on the CAS-8 measure who shifted.

This study summarized the implementation and effectiveness outcomes of the BRAVE Self-Help program, reaching more than 50,000 users over 10 years of delivery. Despite implementation evaluation methods not being prominent over 10 years ago when this program was first implemented, this study offers a retrospective examination of implementation using available data. In doing so, it acknowledges that some information required to assess implementation according to contemporary standards was not collected. Using an open effectiveness-implementation trial design with a large Australian child and adolescent sample, this study represents one of the first demonstrations, within pediatric mental health, of how a publicly available DMHP for child and adolescent anxiety can be implemented nationally using a strategic approach grounded in best-practice implementation frameworks. This platform, delivering an evidence-based therapeutic program, is distinct from other national telephone or digital crisis support services (eg, Lifeline and Kids Helpline), which offer in-the-moment support for distress, rather than complete online CBT interventions. The findings show that BRAVE Self-Help is a scalable, responsive and effective approach for improving anxiety symptoms among large numbers of young people, and has been, and continues to be, able to provide support in times of community- and nationwide stressors.

The findings of this study highlight the importance of involving education and health professionals in any implementation and dissemination efforts for programs involving child and adolescent mental health. Two-thirds of participants were referred into the program via school (eg, guidance counselors or school psychologists) or health professionals, with the latter most likely to be referred by Psychologists. The tendency for people to find the program via professional rather than self-referral highlights the potential for DMHPs to provide service support to mental health professionals, potentially support waitlist reduction, and assist in directing evidence-based support to young people who might otherwise not be able or willing to receive it. Successful implementation of DMHPs for young people in the community will require implementation strategies directly integrating school and health professionals. Although this study did not explicitly assess the impact of its implementation strategies (Multimedia Appendix 1) on implementation outcomes, the strategies were well aligned with recommended effective implementation strategies [18] and could provide examples for future digital models of service delivery, policymakers, and relevant stakeholders.

A key observation was the notable increases in program use aligning with significant community- and nationwide stressors, in particular, during the COVID-19 pandemic and significant weather events. This underscores two critical points: (1) an escalating demand for accessible digital services addressing youth mental health needs, and (2) the ability of the BRAVE Self-Help program to respond to local, national, and global events. This study’s findings are consistent with broader evidence indicating a rise in youth mental health problems that may have subsequently led to a trend of increasing service use, particularly in the context of nationwide stressors [30]. The overall increasing program use trend observed here is also consistent with evidence from other national DMHP platforms in Australia [4,31]. The findings underscore the growing relevance of programs such as the BRAVE Self-Help DMHP in meeting the needs of young Australians, including the potential to address the evolving mental health landscape aggravated by societal challenges.

The results of this study demonstrate that BRAVE Self-Help was able to reach a demographically and clinically diverse cohort of young people. The program reached young people across vast geographic areas, including substantial proportions of young people from regional and rural areas who typically only have limited access to evidence-based mental health services. The program was also able to reach young people with mild, moderate, and severe levels of anxiety, with over half experiencing levels of anxiety that were causing problems for the young person (eg, indicative of an anxiety condition). Though overall there were more female than male users, the program was able to reach male adolescents (30% of adolescent users) and male children (44% of child users). Consistent with school-based research [24], girls presented with higher anxiety symptom severity than boys, and adolescents generally reported higher anxiety severity than children. Importantly, during implementation, the program was able to reach young people identifying as nonbinary or other gender (3.9% of adolescent users), who also presented with notably higher levels of anxiety severity than boys and girls. The proportion of users identifying as other genders increased throughout the extended pandemic (2020‐2022), coinciding with higher levels of anxiety, although the sample size is small. There is a need for further research to explore the suitability of DMHPs for priority populations.

It is worth noting that although adolescent users demonstrated higher levels of anxiety than children over 10 years of service delivery, especially during the extended pandemic, their use of the program reduced in the last 4 years, while child use continued to grow. A potential explanation for reduced use of DMHPs by this cohort could be attributed to prolonged digital fatigue and social isolation through the extended pandemic period, which may have been particularly pronounced for adolescents who are in a developmental stage characterized by an increased need for peer interaction [32]. Moreover, child users may be more likely to have parental assistance in seeking this model of care and adhering to the BRAVE Self-Help program than adolescent users. Therefore, this study offers insights into the specific groups of young people prone to heightened anxiety levels and those more inclined to demonstrate increased program use. These findings can inform future tailored service delivery strategies for disaster preparedness efforts among potentially at-risk groups.

The BRAVE Self-Help program consistently demonstrated acceptability and effectiveness in anxiety symptom reduction for children and adolescents. It was interesting to note that participants referred via school-based professionals had slightly lower satisfaction than those referred by health professionals, or those who self-referred after finding the program online. This may be because schools were using the program as part of curriculum delivery (and therefore may not have been appropriate for some young people) or that they referred young people inappropriately to the program. Although effectiveness was higher for children than adolescents, it is important to note that adolescents also presented to the program with higher anxiety severity, likely entrenched over several years from childhood to adolescence. For example, 63% of children and 50% of adolescents with elevated anxiety at program registration achieved clinically meaningful improvements in their anxiety throughout program participation. This finding is consistent with our previous BRAVE Self-Help effectiveness outcomes, with a comparable medium to large effect size noted for users presenting with elevated anxiety [9]. This study also demonstrates that users with no elevated anxiety showed significant reductions in anxiety following program participation, although this effect was small. Some originally nonanxious participants showed deterioration through the study. It is important to note that BRAVE Self-Help was not developed as a prevention intervention, but rather as an early treatment for existing symptoms of anxiety. Therefore, the sessions and strategies were not targeted at preventing new anxiety development. It is likely that it was difficult for such participants to implement strategies given they did not initially present with anxiety.

Overall, the findings affirm the potential effectiveness of this digital self-help intervention for managing anxiety in young people and further support the role of DMHPs as a viable public health and population-level delivery model. Considering the substantial estimated demand for mental health care among Australian children and adolescents, including those without indication of need (such as those without mental health disorders in the past 12 months or with subthreshold mental health problems) [33], our findings suggest that BRAVE Self-Help has the potential to reach a broader spectrum of young people with varying needs. The platform can offer both preventive and early intervention approaches, leading to effective symptom improvement.

It is crucial to acknowledge the generally lower engagement rates observed in DMHPs, including BRAVE Self-Help. While average completion rates were modest, findings indicate that clinically meaningful reductions in anxiety can occur even with partial program completion, though it is important to acknowledge that this may reflect natural recovery over time and that users may discontinue the program for many reasons. Within the context of a self-directed public health intervention, early disengagement may reflect difficulties progressing through the self-help program on their own; however, it may also reflect symptom improvement or goal attainment rather than treatment failure. Importantly, treatment effects increased with greater program exposure, with the strongest reduction reported among users who completed 6 or more sessions. Identification of young people who require the full treatment program and those who are most likely to disengage for reasons other than symptom improvement will be important in generating clinical recommendations and triaging young people into appropriate services. Importantly, low program engagement for young people is not unique to DMHPs; the average number of sessions attended by young people at youth-focused headspace in-person clinics in Australia is 4.6 sessions, with 53% of participants attending only 1 to 3 sessions [34]. However, even with low engagement, programs such as BRAVE Self-Help can support many young people in achieving symptom reduction; however, the long-term maintenance of such improvements could not be determined in this study. Understanding engagement challenges is crucial for optimizing digital mental health interventions in the future. Strategies that may enhance user engagement have been identified by users of BRAVE-ONLINE, and include personalization of DMHPs, additional professional and community support, and flexible delivery options [35].

This study included a very large sample and conducted a comprehensive evaluation of implementation and effectiveness outcomes of this DMHP including reach, penetration, acceptability, fidelity, and effectiveness. The implementation and sustainability strategies used for BRAVE Self-Help can inform the dissemination of other DMHPs, and the study provides a potentially replicable evaluation framework for other DMHPs in the future. Additionally, the effectiveness of the program was examined via a robust standardized tool for child and adolescent anxiety (CAS-8), facilitating comparisons with national and global norms. Limitations of this study include the use of an observational study design with no control, meaning we cannot be certain that improvements in anxiety are solely attributable to the intervention. It is possible that the improvements in anxiety symptoms reflect natural recovery over time, or regression to the mean, though it was clear that greater program engagement was associated with greater symptom reduction.

In terms of program completion, it is important to acknowledge that engagement differed according to anxiety severity, location, age, referral sources, and gender. Though our nonadherence survey provided some qualitative insights into reasons for noncompletion, this data were limited to a 3-month subsample of users collected very early in program delivery and may not be generalizable to later enrollments in the program in a vastly different DMHP landscape. Further work is required to identify the reasons for program noncompletion among different young people now participating in the program and to provide guidance as to who self-help programs are most appropriate for. It is also important to acknowledge that findings derived from the Australian context may not readily generalize to other countries due to potential cultural, contextual, and health care systems variations, as well as the unique circumstances occurring during the COVID-19 pandemic. Nevertheless, this study involved large-scale nationwide engagement of Australian children and adolescents in a self-help DMHP.

The BRAVE Self-Help program is an accessible, scalable, and effective DMHP that provides a viable public health approach to supporting child and adolescent anxiety. The utility of this program remained stable across periods of cumulative community, national, and global stressors, including the COVID-19 pandemic, with consistent completion, satisfaction, and effectiveness outcomes over 10 years of delivery. Contrary to concerns that digital fatigue [36] may have reduced young people’s engagement with online mental health interventions, findings from this study demonstrate sustained demand, acceptability, and effectiveness of DMHPs such as the BRAVE Self-Help program for anxiety.