This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
There is a high prevalence of children and young people (CYP) experiencing mental health (MH) problems. Owing to accessibility, affordability, and scalability, an increasing number of digital health interventions (DHIs) have been developed and incorporated into MH treatment. Studies have shown the potential of DHIs to improve MH outcomes. However, the modes of delivery used to engage CYP in digital MH interventions may differ, with implications for the extent to which findings pertain to the level of engagement with the DHI. Knowledge of the various modalities could aid in the development of interventions that are acceptable and feasible.
This review aimed to (1) identify modes of delivery used in CYP digital MH interventions, (2) explore influencing factors to usage and implementation, and (3) investigate ways in which the interventions have been evaluated and whether CYP engage in DHIs.
A literature search was performed in the Cochrane Library, Excerpta Medica dataBASE (EMBASE), Medical Literature Analysis and Retrieval System Online (MEDLINE), and PsycINFO databases using 3 key concepts “child and adolescent mental health,” “digital intervention,” and “engagement.” Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed using rigorous inclusion criteria and screening by at least two reviewers. The selected articles were assessed for quality using the mixed methods appraisal tool, and data were extracted to address the review aims. Data aggregation and synthesis were conducted and presented as descriptive numerical summaries and a narrative synthesis, respectively.
This study identified 6 modes of delivery from 83 articles and 71 interventions for engaging CYP: (1) websites, (2) games and computer-assisted programs, (3) apps, (4) robots and digital devices, (5) virtual reality, and (6) mobile text messaging. Overall, 2 themes emerged highlighting intervention-specific and person-specific barriers and facilitators to CYP’s engagement. These themes encompassed factors such as suitability, usability, and acceptability of the DHIs and motivation, capability, and opportunity for the CYP using DHIs. The literature highlighted that CYP prefer DHIs with features such as videos, limited text, ability to personalize, ability to connect with others, and options to receive text message reminders. The findings of this review suggest a high average retention rate of 79% in studies involving various DHIs.
The development of DHIs is increasing and may be of interest to CYP, particularly in the area of MH treatment. With continuous technological advancements, it is important to know which modalities may increase engagement and help CYP who are facing MH problems. This review identified the existing modalities and highlighted the influencing factors from the perspective of CYP. This knowledge provides information that can be used to design and evaluate new interventions and offers important theoretical insights into how and why CYP engage in DHIs.
Mental health (MH) problems in childhood and adolescence are of great importance because of their prevalence, early onset, and impact on different areas of the child’s life [
Prevalence of mental health problems in children and young people.
Mental health problem | Prevalence (%) (adapted from Polanczyk et al [ |
Anxiety | 6.5 |
Disruptive behavior | 5.7 |
Oppositional defiance disorder | 3.6 |
Attention-deficit hyperactivity | 3.4 |
Depression | 2.6 |
Conduct | 2.1 |
Addressing MH problems in CYP is a major public health concern [
The World Health Organization (WHO), the United Kingdom’s National Health Service, and the US National Institute of Mental Health have identified MH apps as cost-effective and scalable solutions for addressing the MH treatment gap [
Despite the growing interest in using mobile apps to deliver interventions, more research evidence is needed to support implementation in children and young people’s mental health (CYPMH) services [
Despite the potential of DHIs, researchers have identified several limitations that influence practicality [
The rapid advancement of technology [
The primary aim of this study was to review the literature to identify modes of delivery used to engage CYP in digital MH interventions. Second, we explored barriers and facilitators for the usage and implementation of DHIs. The authors also aimed to investigate the ways in which these interventions have been evaluated and whether CYP engage in DHI research. The following questions were addressed:
What modes of delivery are used for engaging CYP in digital MH interventions?
What are the barriers and facilitators to engaging CYP in digital MH interventions?
How do retention rates vary in CYP digital MH intervention research?
A literature search was conducted using the Cochrane Library, EMBASE, MEDLINE, and PsycINFO databases. All searches were carried out on the same day (December 27, 2018) to control for daily updates. Overall, 3 key concepts informed the search strategy:
Screened articles were included if (1) the study targeted a CYP sample with a mean age less than 25 years; (2) the article described a DHI targeting an MH symptom (related to a primary physical/somatic condition) or the intervention was being used by CYP with MH problems; and (3) the study explored the development or testing of a DHI resulting in data on adherence, acceptability, or barriers and facilitators to engagement. Any study design was deemed acceptable for inclusion. Articles were excluded if (1) the age of the participants was not defined or if the mean age of the sample was 25 years and above, (2) the intervention was for the sole purpose of communicating between a health care professional and the CYP (eg, Skype, email, teleconference, or messages for appointment reminders), (3) the outcome of the study was not clearly defined or did not provide sufficient details to determine if the outcome was directly related to the intervention, and (4) the study had no human participants (eg, discussion articles describing a novel intervention).
In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart of the study selection process.
A standardized form [
The extracted data were collated and summarized to produce a narrative summary of the study characteristics that addressed the first review question. A descriptive numerical summary was presented to group articles by the primary digital platform used to deliver the intervention. SL completed a qualitative data-driven thematic analysis [
Initially, the review team planned to investigate recruitment rates. However, the identified studies varied in recruitment strategies and did not provide sufficient details to address this research question. In addition, although the review team acknowledged the potential of gray literature (eg, research not published in peer-reviewed journals) to broaden the scope of systematic reviews, the team agreed to only include articles published in peer-reviewed journals. This decision was because of various reasons: (1) the popularity of technological advancements in health care; (2) the resource constraints of this study; (3) some evidence of the limited contribution of unpublished studies to the results of meta-analyses in child-relevant reviews [
The results of this systematic review are presented as a narrative synthesis [
A broad range of recruitment strategies were used to develop and test these DHIs, including referrals from health or school professionals; self-referrals through social media and web-based advertising; university email lists; recruitment software; or in-person advertising through posters, flyers, newspaper advertisements, word of mouth, and existing research and support groups. The following section presents the modes of delivery for DHIs, highlighting how they have been evaluated and their main features and purpose.
Characteristics of the included articles (N=83).
Characteristics | Values, n (%) | ||
|
|||
|
United States and Canada | 31 (37) | |
|
Australia and New Zealand | 23 (28) | |
|
Europe | 21 (25) | |
|
Asia | 7 (8) | |
|
Brazil | 1 (1) | |
|
|||
|
Cognitive behavioral therapy | 39 (47) | |
|
Cognitive skills training mechanisms | 9 (11) | |
|
Social skills training or social support | 7 (8) | |
|
Applied behavior analysis concepts | 3 (4) | |
|
Single component or combinationsa | 25 (30) | |
|
|||
|
Affectiveb | 38 (46) | |
|
Attention deficit and hyperactivity | 7 (8) | |
|
Autism spectrum | 12 (15) | |
|
Eating disorders | 4 (5) | |
|
Behavioral disordersc | 10 (12) | |
|
Nonspecific or multiple disorders | 12 (15) |
aCombinations of the following strategies: therapeutic support embedded in positive psychology, behavior activation, self-regulation, learning theories, motivational interviewing, and mindfulness.
bDepression, anxiety, or suicidality.
cObsessive compulsive disorder, substance abuse, selective mutism, social difficulties, and psychosis or schizophrenia.
Summary of digital modes of delivery used in children and young people’s mental health intervention.
Mode of delivery (number of articles) | Goals: features | Study design, n |
Website interventions (n=43) |
Communication: emails, text messages, social networking, web-based message boards, discussion forums Dissemination of information: text and multimedia channels (videos, animations, and audio), games and quizzes, homework tasks, and web-based profile set up with customizations |
RCTa (n=22), pre- to posttest (n=11), observational study—qualitative, quantitative, or mixed methods approaches (n=10) |
Games or computer-assisted interventions (n=23) |
Dissemination of information, skills development, psychoeducation: photos, stories, animations, quizzes, text messages, and multimedia (audio and videos) |
RCT (n=11), pre- to posttest (n=8), observational study—quantitative or mixed methods approaches (n=4) |
Apps: web or mobile (n=10) |
Dissemination of information, skills development, peer-to-peer communication: text message reminders, text, photos and multimedia (audio and videos) plus an opportunity to upload content |
RCT (n=4), pre- to posttest (n=1), observational study—qualitative, quantitative, or mixed methods approaches (n=5) |
Robots and digital devices (n=3) |
Dissemination of information Peer-to-peer communication: email reminders |
RCT (n=1), feasibility study (n=1), mixed methods design (n=1) |
Virtual reality experiences (n=3) |
Dissemination of information, skills development, therapeutic support: Gamification using multimedia (audio and images) |
Pre- to posttest (n=2), posttest (n=1) |
Mobile text messages (n=1) |
Dissemination of information, skills development, social support: Text |
RCT (n=1) |
aRCT: randomized controlled trial.
Overall, 33 of the 43 articles adopted an interventional study design (22 RCTs and 11 pre- to poststudy designs). The remaining 10 studies adopted observational study designs utilizing qualitative, quantitative, or mixed methods approaches. The methodological quality of the included studies was acceptable. Qualitative studies scores ranged from 2 to 5 points, RCTs and nonrandomized quantitative studies also ranged 2 to 5 points, and mixed methods studies ranged 2 to 4 points.
The primary goal of the majority of the interventions (n=40) was to transmit specific MH information to a targeted population. In addition, 12 of the 40 articles described interventions that were multipurpose, providing an additional opportunity for peer communication (n=7) or for personal health tracking (n=4). YouthCHAT (see
Various features were adopted to achieve the above goals. Communication occurred digitally using emails, text messages, social networking, web-based message boards, and discussion forums. Dissemination of information occurred through text and multimedia channels (eg, videos, animations, and audio). Some interventions also utilized games and quizzes, homework tasks, and a web-based profile set up with customizations.
Overall, 20 of the 23 articles adopted an interventional study design (11 RCTs and 8 pre- to poststudy designs). The remaining 4 studies adopted observational study designs utilizing quantitative or mixed method approaches. The methodological quality of the included studies varied. RCTs scores ranged 2 to 5 points, nonrandomized quantitative studies ranged 2 to 4 points, and mixed methods ranged 3 to 5 points. No articles used qualitative methods only.
The primary goal of the majority of the interventions (n=18) was to transmit specific MH information to a targeted population. Of the 18 interventions, 4 were multipurpose, providing additional general information to the public (n=1) or an opportunity for personal health tracking (n=4). In addition, 8 interventions focused on cognitive training tasks. The computer-assisted instruction intervention was used as a facilitator to assist children in developing reading skills. The social stories accessed via tablets were also used for psychoeducational purposes.
The gamification approach used to achieve the above goals was accessed either on the web or offline and incorporated photos, stories, animations, quizzes, text messages, and videos.
Of the 10 articles, 5 adopted an interventional study design (4 RCTs and 1 pre- to poststudy design). The remaining 5 articles adopted observational study designs utilizing qualitative, quantitative, or mixed methods approaches. The methodological quality of the included studies varied. Qualitative studies scored either 4 or 5 points, RCTs scores ranged 1 to 4 points, the 1 nonrandomized quantitative study scored 4 points, whereas the 2 mixed methods studies scored 3 points.
The primary goal of most apps was to transmit specific MH information to a targeted population. Furthermore, 3 apps were multipurpose, providing an additional opportunity for personal health tracking. The TECH app further included peer-to-peer communication.
The included apps were either web apps or mobile apps and included text message reminders, text, photos, and multimedia (videos and audio). Users also had the opportunity to upload content such as videos and photos.
Of the 3 studies, 2 adopted an interventional study design, of which one (CommU) was an RCT. The study on the Fitbit Flex and Facebook adopted a mixed methods design. The study on ARIA adopted an observational study design as a pilot usability study. The methodological quality of the included studies was acceptable. The ARIA study scored 3 points, CommU scored 3 points, and the Fitbit Flex and Facebook intervention study scored 5 points.
The primary goal of ARIA and CommU was to transmit specific MH information to a targeted population, whereas the Fitbit Flex and Facebook intervention additionally provided an opportunity for peer communication.
ARIA and CommU utilized audio and movement to achieve the above purpose. The Fitbit Flex and Facebook synced with 2 other approaches, an app and a website, and included email reminders to achieve its purpose.
Furthermore, of the 3 studies, 2 interventions adopted pre- to posttest designs, whereas one adopted a posttest-only design (Virtual Dolphin Interaction). No RCTs were found to evaluate the identified interventions. Cave Automatic Virtual Environment also incorporated a mixed methods approach and obtained qualitative data. The methodological quality was acceptable. The Cave Automatic Virtual Environment and Collaborative Virtual Environment studies scored 3 points and the Virtual Dolphin Interaction study scored 4 points.
The primary goal of all 3 interventions was to transmit specific MH information to a targeted group or to facilitate skills training or provide therapeutic support. Collaborative Virtual Environment also functioned as a communication aid to facilitate collaboration within the virtual reality environment.
Cave Automatic Virtual Environment, Collaborative Virtual Environment, and Virtual Dolphin Interaction utilized features of the gamification approach to engage CYP in a more real-life experience, allowing for more immersion and movement.
In addition, 1 text messaging intervention was identified and evaluated in an RCT. The methodological quality score was 3. The Educating and Supporting Inquisitive Youth in Recovery program aimed to transmit specific MH information to a targeted audience, to facilitate skills training or offer therapeutic support, to provide the opportunity for personal health tracking, and to signpost CYP to additional social support websites. No additional features, apart from text, were described in the study. However, participants were contacted via phone as part of the study.
Influencing factors presented as barriers and facilitators to engagement emerged as 2 broad themes encompassing 6 factors: intervention-specific influences (suitability, usability, and acceptability) and person-specific influences (motivation, capability, and opportunity). Overall, 29% (24/83) of the included articles provided data to support these themes, 13 studies provided data for suitability, 13 for usability, and 14 provided data for acceptability. Of the 24 articles, 8 provided data to inform motivation, 4 for capability, and 13 for opportunity. A summary of concepts corresponding to the individual factors within the major themes is presented in
A framework of factors influencing engagement in children and young people’s mental health digital intervention.
CYP highlighted factors related to the development of the intervention, which influenced whether they used the intervention or not. A prominent factor influencing the acceptability (ie, willingness to use) of the intervention was the features, whereby CYP highlighted certain images, specific language, and interfaces that were unappealing to them. They made suggestions and highlighted features such as videos, having less text, ability to personalize or create a profile, and ability to connect with others or receive text message reminders as encouraging their use of the intervention. CYP also suggested that providing rewards could also be a motivating factor for engaging with DHIs. Similarly, usability (ie, the degree to which the DHI was able to be used) was important for promoting engagement. Interventions that CYP favored were described as self-paced, user friendly, age appropriate, simple, and straightforward. However, in situations where CYP had problems understanding the task, or if the intervention did not provide sufficient instructions on usage, they were less inclined to continue using the DHI.
Another main factor was the suitability (ie, the degree to which the DHI is in line with daily activities) of the intervention to the lifestyle of CYP. Although CYP liked not having to travel to access the intervention and the ability to use it while at home, they were
Of the 6 factors, 3 were associated with person-specific barriers and facilitators to behavior change, which is in line with the COM-B model [
The second major factor identified as a person-specific influence was motivation. Some tasks were of less interest to CYP, and sometimes, they would have forgotten the existence of the intervention. However, they highlighted that curiosity and perceived need influenced their usage. Perceived usefulness of the intervention to address their needs was a major motivating factor; therefore, if the resource was viewed as unhelpful or too general, CYP were less interested in using it. However, although CYP were eager to use DHIs, the capability to engage with the intervention was sometimes affected by physical, environmental, and MH stressors, representing another major influencing factor.
Owing to the heterogeneity in study design and intervention type, not all articles provided sufficient detail to estimate retention rates. Therefore, we were unable to include estimates for 16 of the included articles. The average retention rate across the remaining 67 articles showed almost 80% of CYP using DHIs or completing the follow-up measures. Results showed that the retention rates varied, with 11.9% (8/67) of articles achieving 100% retention and 8.95% (6/67) reporting less than 50% retention. Overall, 83.58% (56/67) of the included articles had a retention rate of at least 70%. Subgroup analyses for approaches with a larger number of articles indicated that the average retention rate for games and computer-assisted intervention studies was 86.95%, followed by websites interventions with 78.87%, and apps with 78.45%.
This review identified 6 modes of delivery of DHIs for CYP with MH issues: websites, games and computer-assisted programs, apps, robots and digital devices, virtual reality, and mobile text messaging. Overall, 2 themes emerged, highlighting intervention-specific and person-specific barriers and facilitators to engagement in DHIs. In addition, the findings of this review suggest a high average retention rate of almost 80% when the identified modalities were evaluated. Knowledge of these approaches, including influencing factors to usage from the perspective of the CYP, provided information that can be used to design and evaluate new DHIs.
From the articles reviewed, 59 contributions were published between 2013 and 2018, with 15 records published in 2018. This is consistent with the fact that interest in applying digital technologies to MH practice has been increasing since the early 2000s, and recommendations for research in this field were issued only in 2013 [
The modes of delivery identified in this review are similar to those identified in other reviews exploring adolescent physical health [
Other barriers and facilitators that were identified in this review also emphasize the importance of user-centered design methods when developing DHIs for CYP [
Our findings suggest that the retention rate of CYP in digital MH interventions (mean retention rate of 79.2%) was superior to that reported in face-to-face CYP MH outpatient care, where dropout affects 20% to 60% of the cases [
DHIs were included at various stages of the provision of psychological support. Technology-mediated programs and tools were part of prevention, assessment, treatment (psychoeducation and psychotherapy) and follow-up of MH care. This extensive potential of DHIs can support the WHO’s initiative to identify and intervene to lessen the MH treatment gap [
DHIs can be a helpful way to support and treat MH problems. Such tools can complement the various stages of the provision of psychological support or psychotherapy among CYP with MH problems. However, effective implementation and sustained usage will rely on the extent to which the design is appropriate for the intended purpose and how it will be used in practice. This understanding may help to minimize the risks associated with fear of usage that some end users experience by providing useful directions on how to design technologically responsible therapeutic approaches [
Although this study provides insight that is valuable for the development of new interventions, future research should (1) not only focus on the effectiveness but also investigate engagement, taking into account influencing factors, as an important component of research; (2) arrive at a consensus on defining engagement and how it should be measured; (3) provide adequate reporting of recruitment and retention rates; and (4) compare CYP preferences for various modes of delivery or therapeutic approaches. Finally, this study also acknowledges the implications for practice. The findings suggest CYP interest in DHIs, and therefore, (1) efforts to improve engagement may be beneficial to CYPMH outcomes and (2) families including the CYP and clinicians should work together to identify DHIs that are suitable to the CYP’s lifestyle.
This review adhered to established guidelines for systematic reviews [
DHIs may be of interest to CYP, particularly in the area of MH treatment. Research on retention rates suggests high engagement of CYP in digital MH interventions that may encourage further development of DHIs in the near future. CYPMH services could benefit from this development as the included studies indicate. However, the results of this review highlighted intervention-specific and person-specific factors that influence CYP usage of digital MH interventions that should be considered. With continuous technological advancements, it is desirable to know which modalities may increase usability and adherence to better support CYP facing MH challenges.
Search strategy.
Reviewed articles.
Intervention characteristics.
Characteristics of reviewed articles.
Themes and supporting codes.
capability, opportunity, motivation, and behavior
children and young people
children and young people’s mental health
digital health intervention
mental health
mixed methods appraisal tool
randomized controlled trial
World Health Organization
This study was funded by the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement number 722561; TREATme, a European Network on Individualized Psychotherapy Treatment of Young People with Mental Health Disorders under the European Cooperation in Science and Technology which is supported by the EU Framework Programme Horizon 2020 action CA16102; European COMPETE2020 (POCI-01-0145-FEDER-030980); Estonian Research Council grant PUT1518; and Portuguese National funds FCT—Fundação para a Ciência e a Tecnologia, I P (PTDC/PSI-ESP/30980/2017).
SL and JE-C conceptualized the paper. SL, CPM, CMDS, AC, S Carletto, CH, SS, S Cerón, PMP, GP, BM, and JE-C carried out selection of articles, data extraction, data analysis, and writing of the original draft. SL, CPM, CMDS, AC, S Carletto, CH, SS, S Cerón, PMP, GP, BM, RU, NF, and JE-C were involved in the writing, review, and editing.
JE-C is an author of one of the reviewed articles.