Published on in Vol 22, No 10 (2020): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/18672, first published .
The Role of Technology and the Continuum of Care for Youth Suicidality: Systematic Review

The Role of Technology and the Continuum of Care for Youth Suicidality: Systematic Review

The Role of Technology and the Continuum of Care for Youth Suicidality: Systematic Review

Authors of this article:

Hannah Szlyk 1 Author Orcid Image ;   Jia Tan 2 Author Orcid Image

Review

1School of Social Work, Rutgers, The State University of New Jersey, New Brunswick, NJ, United States

2The Brown School, Washington University in St Louis, Saint Louis, MO, United States

*all authors contributed equally

Corresponding Author:

Hannah Szlyk, MSW, PhD

School of Social Work

Rutgers, The State University of New Jersey

390 George Street

New Brunswick, NJ, 08901

United States

Phone: 1 9178364225

Email: hannah.szlyk@rutgers.edu


Background: Youth suicide is a global public health issue, and using technology is one strategy to increase participation in preventive interventions. However, there is minimal knowledge on how technology-enhanced interventions for youth correspond to the stages of care, from illness or risk recognition to treatment follow-up.

Objective: This systematic review aims to examine the efficacy of technology-enhanced youth suicide prevention and interventions across the continuum of care.

Methods: Four electronic databases were searched up to spring 2019 for youth suicide preventive interventions that used technology. The review was not restricted by study design and eligible studies could report outcomes on suicidality or related behaviors, such as formal treatment initiation. An adapted version of the Methodological Quality Ratings Scale was used to assess study quality.

Results: A total of 26 studies were identified. The findings support the emerging efficacy of technology-enhanced interventions, including a decline in suicidality and an increase in proactive behaviors. However, evidence suggests that there are gaps in the continuum of care and recent study samples do not represent the diverse identities of vulnerable youth.

Conclusions: The majority of identified studies were conducted in school settings and were universal interventions that aligned with the illness and risk recognition and help-seeking stages of the continuum of care. This field could be strengthened by having future studies target the stages of assessment and treatment initiation, include diverse youth demographics, and examine the varying roles of providers and technological components in emerging interventions.

J Med Internet Res 2020;22(10):e18672

doi:10.2196/18672

Keywords



Background

Youth suicide is a global public health crisis. In the United States, suicide is the second leading cause of death for children and youth aged 10-24 years [1]. Globally, suicide is the third leading cause of death for youths aged 15-19 years [2]. In addition to deaths by suicide, suicidality includes suicidal ideation and related behaviors, such as plans to attempt suicide and actual suicide attempts [3]. Thus, research suggests that the risk of youth suicide may even be more pervasive as many youths experience suicidal ideation and nonfatal suicidal behaviors [4]. Universally, adolescence and early adulthood are vulnerable periods for when suicide risk is particularly elevated [5].

Therefore, it is important that youths have access to global systems of mental health care. Mental health services and resources may range from promoting illness recognition (in the case of suicidality, this may include risk factor recognition) to providing targeted treatment and to offering follow-up services. This range, which spans the levels of intensity in care, is often called the continuum of care [6,7]. Suicidal individuals who are engaged in an integrated continuum of mental health care may experience decreases in suicidality [8]. However, youth engagement in the continuum of mental health care is often complicated as a consequence of developmental changes, the delayed detection of symptoms, and delayed access to treatment [9]. Thus, researchers need to ensure that available interventions are tailored to specifically meet youths’ needs and correspond with the stages of the continuum of care [9].

Technology is one of the identified mediums to bridge gaps in the continuum of suicide preventive interventions [10]. Technology is especially relevant to engaging youths around the world. Research suggests that most young people in the United States and in developed countries have access to smartphones [11,12], whereas access increases among younger cohorts of emerging economies [12]. Therefore, the use of technology may address barriers to face-to-face care, such as access, reach, and stigma [10,13]. Technology-enhanced interventions use technology to solely deliver or serve as a component of an intervention and can include a mobile phone app, text messaging, telephone, videos, and web-based platforms [13]. Previous reviews on this topic have been restricted to gatekeeper interventions [14], including interventions across the lifespan or interventions designed to address broad mental health issues [15], focused on specific technologies [16], or that may be outdated as new interventions have since been developed [17]. No known review has explored how current technology-enhanced suicide interventions for youth correspond to the stages of the continuum of care. Thus, there is an incomplete understanding of the breadth and efficacy of preventive interventions that use technology and serve youth at risk of suicide.

Objectives

To address these gaps in the literature, this systematic review aims to examine the efficacy of technology-enhanced youth suicide prevention and interventions across the continuum of care. The authors evaluated study outcomes in addition to suicidality, including help-seeking behaviors and coping skills, to better assess how the literature supports youth in leading lives worth living. The findings have implications for how suicidology may address identified gaps in the stages of the continuum of technology-enhanced suicide interventions and enhance care for vulnerable youths.


Search Strategy

The search was conducted in spring 2019, and the systematic review adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (Figure 1) [18].

Electronic databases were searched (PsycINFO, CINAHL [Cumulated Index to Nursing and Allied Health Literature], Ovid MEDLINE, ClinicalTrials.gov) using search terms specific to youth, suicidality, technology, and interventions (Multimedia Appendix 1). The study selection and data extraction were conducted by 2 assessors (the first and second authors). Eligible studies were required to focus on youths who were at potential risk, at high risk, or struggling with suicidality, be in English, and have the majority of participants between the ages of 12 and 24 years [19,20]. Selected studies adhered to the definition of technology-enhanced interventions (as previously defined) by Kreuze et al [13]. The investigators decided not to exclude studies that had primary outcome variables other than suicidality. This decision was based on a preliminary search of the literature, in which the investigators noticed that studies of technology-enhanced interventions for youth fell into several primary outcome domains (see the Data Extraction section for the outcome domains explored). The intention was to identify studies that may have been overlooked in previous reviews restricted to outcomes of suicidality and that applied to the stages of the continuum of care. Relevant systematic reviews were also cross-checked to adjust initial search terms and to potentially identify studies that had been missed in the final search.

Studies that focused on assisted suicide, nonsuicidal self-injury, postvention, or only gatekeeper outcomes were excluded from the final sample. In addition, the investigators excluded studies that only used technology to collect information about participant characteristics and behaviors, which only provided qualitative results, or only discussed the psychometrics of their assessment tool. No exclusion was placed on the trial design. When multiple publications of the same intervention were identified, the most recent or the most advanced trial was selected. Preliminary searches were organized using Endnote [21], and screening and data extraction were conducted using Covidence [22] and spreadsheets.

Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram for search strategy.
View this figure

Data Extraction

Data were extracted for study characteristics (Tables 1 and 2). As gender bias in peer review and in first author publications exists [23,24], the first author’s gender identity was extracted and determined by searching department profiles. In addition, as suicidology is considered a multidisciplinary field, the investigators decided to document the first author’s field of degree by searching web-based profiles and curriculum vitae. Studies’ stages on the continuum of care were also extracted, and studies could cover more than 1 stage. The continuum of care framework used in this review was based on the literature from various health disciplines [25-27] and included the following stages: illness or suicide risk factor recognition, help seeking, assessment, treatment initiation, treatment module, treatment engagement, and follow-up.

Quality of study design and methodology were assessed using an adapted version of the Methodological Quality Rating Scale (MQRS; see Multimedia Appendix 1 for full scale) [28]. The MQRS was originally developed to review alcohol treatment outcomes and covered 12 domains, such as study design, documentation of quality control of treatments, and reports of participants lost to attrition [28,29]. Each domain is rated by the study’s strategies to decrease bias, with studies reporting more rigorous strategies receiving a higher quality score. The MQRS has been used to review other substance and mental health intervention studies [30,31] and has been adapted to evaluate the inclusion of theoretical frameworks and cultural linguistic adaptions in study development [32,33]. The investigators assessed interrater reliability by calculating the percent agreement for each MQRS domain. For example, there was an 81% agreement on both investigators’ ratings by the domain of study design (see Multimedia Appendix 1 for the percent agreement for all MQRS domains).

The outcome attainment of each selected study was evaluated using a categorization system for statistical significance and outcome direction [33]. Outcomes were grouped into the following domains: suicidality (ie, suicidal ideation, planning suicide attempts, and attempts), co-occurring mental health issues (ie, anxiety, depression), youth’s perceptions and knowledge about suicide, help-seeking behaviors, proactive coping behaviors, and formal treatment initiation, which is the official beginning of participation in outpatient or inpatient mental health treatment.

The investigators documented whether the outcomes related to these domains demonstrated statistically significant changes (at least P<.05) and whether the change occurred in the desired direction (ie, a decrease in suicidal ideation) [33]. Differences in coding and ratings during the phases of study screening and data extraction were resolved through consensus between the 2 investigators.


Study Characteristics

The final sample included 26 studies (Table 1) [34-59]. Although various professional disciplines were represented (ie, social work, medicine, public health), over half (n=16) of the first authors held a doctoral degree in psychology. In total, 16 of the first authors were female; 13 studies were conducted in the United States. Among studies that provided participant ages, the mean or median age ranged from 14.7 to 23 years. Female participants represented the majority of the sample participants. Of the studies that did report youth ethnicity, the majority of participants identified as White; no study reported the sexual or gendered minority identity of the participants. A total of 13 studies were conducted in an educational setting.

The selected studies included a variety of preventive interventions (Table 2); the SOS (Signs of Suicide) model was included in 2 studies: the original model for high schools [34] and the adapted model for middle schools [41]. The modalities of technology most often used were phones and web or web-based platforms. Half of the studies (n=13) described indicated interventions for youth suicidality. A total of 12 studies were randomized controlled trials (RCTs). The outcomes of the adapted MQRS tool ranged from 0 (lowest quality) to 18 (highest quality), with an average score of 10.4 and range of 3 to 16.

Table 1. Study characteristics.
Authors (reference)First author’s fieldCountrySettingSample size, nAge (years), mean (SD)aMajority gender of sampleMajority ethnicity of sample
Universal

Aseltine et al [34]SociologyUnited StatesHigh school2100Not availableFemaleHispanic, non-White

Bailey et al [35]Behavioral health sciencesAustraliaHigh school12916.7 (N/Ab)MaleWhite

Freedenthal [36]Social workUnited StatesHigh school14615.8 (1.2)FemaleHispanic, non-White

Haas et al [37]PsychologyUnited StatesUniversity; web based1162Not availableFemaleNot available

Han et al [38]Public healthAustralia; ChinaUniversity; web based257Subsamples:
  • 18.6 (1.02)
  • 20.1 (2.08)
FemaleChinese

Pisani et al [39]PsychologyUnited StatesRural high school42Not availableFemaleWhite

Robinson et al [40]PsychologyAustraliaHigh school6916.4 (N/A)Not availableNot available

Schilling et al [41]Public healthUnited StatesMiddle school386Not availableFemaleWhite

Wyman et al [42]PsychologyUnited StatesHigh school2675Not availableFemaleWhite
Selective

Dickter et al [43]PsychologyUnited StatesHospital; web based8317.5 (2.04)FemaleWhite

Hetrick et al [44]PsychologyAustraliaCommunity mental health clinic10118.7 (2.8)FemaleNot available

Iorfino et al [45]PsychologyAustraliaPrimary mental health clinic; web based23220.4 (2.59)FemaleNot available

King et al [46]PsychologyUnited StatesUniversity; web based7622.9 (5.0)FemaleWhite
Indicated

Bertolote et al [47]Clinical sciences; psychiatryBrazil, India, Sri Lanka, Iran, ChinaEmergency department186723c median ageFemaleIndian

Czyz et al [48]PsychologyUnited StatesHospital3615.4 (1.36)FemaleWhite

Hetrick et al [49]PsychologyAustraliaHigh school; web based5014.7 (1.4)FemaleNot available

King et al [50]PsychologyUnited StatesHospital44815.6 (1.31)FemaleWhite

King et al [51]PsychologyAustraliaHotline or counseling center101Not availableFemaleNot available

Mehlum et al [52]Medicine or psychologyNorwayChild and adolescent psychiatric outpatient7715.6 (1.5)FemaleNorwegian

Normand et al [53]PsychiatryFranceHospital173Subsamples:
  • 17.9 (1.9)
  • 18.4 (1.8)
FemaleNot available

O’Brien et al [54]Social workUnited StatesPsychiatric outpatient2015.7 (1.6)FemaleWhite

Rosenbaum et al [55]PsychiatryUnited StatesEmergency department18114.7 (2.0)FemaleHispanic, non-White

Tan et al [56]PsychologyChinaWeb based72521.2 (3.69)FemaleNot available

Yen et al [57]PsychologyUnited StatesPsychiatry inpatient unit2015.9 (1.5)FemaleWhite

Yen et al [58]PsychologyUnited StatesPsychiatry inpatient unit5015.7 (1.53FemaleWhite
All tiers

Silverstone et al [59]PsychiatryCanadaMiddle and high school6651Not availableFemaleNot available

aFor mean age, full sample information has been provided; subsample information was reported when the mean and SD of the full sample could not be generated due to insufficient information.

bN/A: not applicable.

cAuthors only provided median age across study sites.

Table 2. Study characteristics continued.
Authors (references)DesignIntervention nameDescriptionTechnology usedMQRSa score (0-18)
Universal

Aseltine et al [34]RCTbSOScAims to raise awareness of suicide risk and promote help seeking for youth and peers. Curricula delivered via video and discussion. Participation in program over 2 daysVideo14

Bailey et al [35]Pre, posttestsafeTalkA 1-time, 3-hour workshop delivered to students by a trainer and school staff. Uses presentations, video, discussion, questions, and role plays. Teaches students about suicide risk, perceptions about suicide, and help-seeking strategiesVideo, phone10

Freedenthal [36]Quasi-experimental trialYellow RibbonA 60-min student leadership training for students and school staff conducted by program trainers. Training is focused on warnings signs of suicide among youth, myths about suicide, and the importance of seeking help for peers or for oneself. Content is delivered via a slide show and Ask4Help cardsDigital slide presentation11

Haas et al [37]Prospective cohort studyCollege screeningAn interactive, web-based program for university students officially called the College Screening Project. The web-based screening identified at-risk students, supported them in getting help, and helped to determine the proportion of students who entered treatmentWeb-based screener, email, web-based chat9

Han et al [38]RCTProHelpA brief, 2-module web-based psychoeducational program that aims to teach students about risk factors for suicide, stigmatizing attitudes, and barriers to help seekingWeb-based platform12

Pisani et al [39]Field testText4StrengthAn automated, interactive text messaging intervention developed for early adolescents in rural communities. It is an extension of the Sources of Strengths program. Youth received messages over 9 weeks that were related to topics of emotion regulation, social connections, and help seekingVideo, text messaging11

Robinson et al [40]Pilot studySocial Media MessageA social media message intervention that was developed by youth for at-risk peers. Designed to increase youth awareness about suicide, risk factors, and strategies to help peers and themselves. Participants evaluated 2 social media messagesVideo, phone, tables and computers, web based3

Schilling et al [41]RCTSOS-Middle SchoolSimilar to the SOS high school version. Features a 17-min DVD that includes 3 age-appropriate vignettes; a group discussion by middle school students about depression, suicide, bullying, self-injury, and getting help; and a student interview with a school-based counselor to model getting help. Delivered by trained school personnelDVD or video13

Wyman et al [42]RCTSources of StrengthAims to improve youth help-seeking behaviors and proactive coping to reduce the risk of suicide. The program has 3 standard phases: (1) school and community preparation, (2) peer leader training, and (3) schoolwide messaging through video and text messaging. Premise is that peer and staff training (varying from 1 to 6 hours) in curriculum encourages sustainability of the programVideo, text messaging11
Selective

Dickter et al [43]Nonrandomized trial, 2 treatment groupsCATCH-IT: The Competent Adulthood Transition with Cognitive Behavioral and Interpersonal TrainingThe Competent Adulthood Transition with Cognitive Behavioral and Interpersonal Training consists of 14 self-guided, web-based modules that use techniques from CBTd and interpersonal psychotherapy. Aims to teach skills for increasing resiliency against depressive disorders and suicidalityWeb-based platform9

Hetrick et al [44]Prospective cohort studyMonitoring ToolA web-based tool for self-monitoring of depression and suicidal ideation that tracked changes in symptoms and alerted clinicians. Participating youth completed the tool between 2 and 8 times (duration varying between 4 and 83 days)Web-based platform10

Iorfino et al [45]Prospective cohort studySynergy Online SystemAn initial clinical assessment on the web before a face-to-face or web-based clinical appointment. The initial clinical assessment assesses a range of mental health outcomes (14 modules, approximately 45 min to complete). At the end of the suicidality module, the algorithms assess current and past suicidality, which alerts clinical staff if the current suicide risk is highVideo, web-based platform6

King et al [46]RCTEBridgeThe Electronic Bridge to Mental Health Services is a web-based screening and intervention program for college students at risk for suicide. The program provides students with feedback from the screening and information about resources and can link students with web-based counseling services. The program aims to increase help-seeking and eventual use of mental health services. Length of program depended on level of student interactionWeb-based platform11
Indicated

Bertolote et al [47]RCTBICA brief educational intervention with periodic follow-up contacts for suicide attempters conducted at global emergency departments and was part of the WHOe Multisite Intervention Study on Suicidal Behaviors (SUPRE-MISS). The BIC procedure includes a standard 1-hour individual information session at the time of discharge. Follow-up contacts by health professionals were 1 week; 2, 4, 7, and 11 weeks; and 4, 6, 12, and 18 months after dischargePhone17

Czyz et al [48]RCTMI-SafeCopeA motivational interview-enhanced safety planning intervention for teens hospitalized because of suicide risk. The intervention includes 3 components: an individual session, a family session, and a 2-week postdischarge booster call by phone (with the intervention counselor). Youth also provided assessments via text message at 1 and 3 months after dischargePhone, text messaging14

Hetrick et al [49]RCTReframe-ITAn internet-based CBT program that aims to reduce suicide-related behaviors, depression, anxiety, and hopelessness and improve problem solving and cognitive and behavioral issues. The intervention consisted of 8 modules of CBT delivered on the web over 10 weeksWeb-based platform12

King et al [50]RCTYST-IIThe Youth-Nominated Support Team–Version II for suicidal adolescents provides psychoeducation and ongoing consultation for the parent-approved adult support persons that have been nominated by the adolescent. The support persons maintain regular supportive contact with the adolescents via phone for 3 months following hospitalizationPhone14

King et al [51]Pre-post testsKids Help LineTrained help line counselors assessed changes in suicidality and mood for youth callers at the beginning of the session and at the conclusion of the phone session. Mean duration of calls was 40 min (range 10-120 min)Phone4

Mehlum et al [52]RCTDBT-A: Dialectical Behavior Therapy-AdolescentDialectical behavior therapy for adolescents lasts from 3 to 5 months, includes parents or other caregivers in weekly skills training groups, and has a skills module to support teens with emotion dysregulation and their families. This trial was delivered over 18 weeks and was delivered by trained mental health professionals. Coaching sessions were delivered over the phonePhone16

Normand et al [53]Prospective cohort study4-Phone-CallsHospital staff called the youth 1 week, 1 month, 6 months, and 12 months after discharge for a suicide attempt. The interviews during the phone calls included informal and formal assessment of current symptoms and the youth’s safetyPhone7

O’Brien et al [54]Pilot studyCrisis CareA smartphone app intervention developed specifically for suicidal adolescents and their parents to use after discharge from the hospital. The app provides access to coping skills and immediate access to help, if neededSmartphone app4

Rosenbaum et al [55]RCTFISPThe Family Intervention for Suicide Prevention is developed for youth admitted to emergency departments post–suicide attempt. Youth and their families participate in a CBT session aimed to increase motivation for follow-up treatment and safety postdischarge. Participants also received structured phone calls 48 hours and often 1, 2, and 4 weeks postdischarge to promote outpatient treatment attendancePhone11

Tan et al [56]Pilot studyMicroblog InterventionAn intervention developed for users of the Sina Weibo microblogging platform. Participants received direct messages designed to respond to high suicide risk postings. The intervention aimed to increase help seeking for at-risk users and peersWeb-based platform; Sina Weibo5

Yen et al [57]Pre-post testSTEPIncludes an in-person phase (4 sessions) and a remote delivery phase (text messaging and phone calls). The inpatient sessions focus on psychoeducation and coping skills. The remote delivery phase consists of weekly phone calls and daily text messages to provide skills practice reminders and to monitor moodPhone, text messaging12

Yen et al [58]RCTCLASP-AThe Coping Long Term with Active Suicide Program for Adolescents program is adapted for adolescents hospitalized for suicidal ideation or a suicide attempt. The program includes 3 individual sessions and 1 family session and a series of outpatient phone calls to adolescent and a designated parent or guardian over 6 months of follow-up postdischargePhone15
All tiers

Silverstone et al [59]Pre-post and follow-upEMPATHYThe multimodal program includes repeated data collection, identification of a high-risk group, a rapid intervention for the high-risk group (a supervised web-based CBT program), a universal CBT intervention, interactions with trained staff, and referrals to external medical and psychiatric servicesWeb-based platform9

aMQRS: Methodological Quality Rating Scale.

bRCT: randomized controlled trial.

cSOS: Signs of Suicide.

dCBT: cognitive behavioral therapy.

eWHO: World Health Organization.

Regarding the continuum of care (Table 3), most studies were targeted to increase illness or risk factor recognition (n=11), to increase help seeking (n=10), and to guide youths through a treatment module (n=10; Table 2). The majority of studies (n=18) addressed >1 stage of the continuum of care. For example, 8 of the 9 universal interventions addressed illness or risk recognition and help seeking; 5 of the 11 indicated interventions focused on the stages (at least) of participating in the treatment module and treatment engagement. As illustrated in Table 4, among the 6 common outcome domains measured, most studies (n=22) reported suicidality (ie, ideation, attempts) as an important study variable, followed by co-occurring mental health issues (n=12; ie, depression, distress, or anxiety).

Table 3. Studies and their stages on the continuum of care.
Authors (references)Illness or risk recognitionHelp seekingAssessmentTreatment initiationTreatment moduleTreatment engagementFollow-up
Universal

Aseltine et al [34]aN/AbN/AN/AN/AN/A

Bailey et al [35]N/AN/AN/AN/AN/A

Freedenthal [36]N/AN/AN/AN/AN/A

Haas et al [37]N/AN/AN/AN/AN/A

Han et al [38]N/AN/AN/AN/AN/A

Pisani et al [39]N/AN/AN/AN/AN/A

Robinson et al [40]N/AN/AN/AN/AN/A

Schilling et al [41]N/AN/AN/AN/AN/A

Wyman et al [42]N/AN/AN/AN/AN/A
Selective

Dickter et al [43]N/AN/AN/AN/AN/AN/A

Hetrick et al [44]N/AN/AN/AN/AN/AN/A

Iorfino et al [45]N/AN/AN/AN/AN/AN/A

King et al [46]N/AN/A
Indicated

Bertolote et al [47]N/AN/AN/AN/AN/A

Czyz et al [48]N/AN/AN/AN/A

Hetrick et al [49]N/AN/AN/AN/AN/A

King et al [50]N/AN/AN/AN/AN/AN/A

King et al [51]N/AN/AN/AN/AN/AN/A

Mehlum et al [52]N/AN/AN/AN/AN/A

Normand et al [53]N/AN/AN/AN/AN/AN/A

O’Brien et al [54]N/AN/AN/AN/AN/AN/A

Rosenbaum et al [55]N/AN/AN/AN/AN/A

Tan et al [56]N/AN/AN/AN/AN/AN/A

Yen et al [57]N/AN/AN/AN/A

Yen et al [58]N/AN/AN/AN/A
All tiers

Silverstone et al [59]N/AN/AN/A
Total, n1110431077

aThe study addresses that stage of the continuum.

bN/A: not applicable.

Table 4. Measured and significant intervention outcomes.
Authors (references)SuicidalityCo-occurring mental health issuesPerceptions and knowledge of suicideHelp seekingCoping behaviorTreatment initiationSignificant outcomes
Universal

Aseltine et ala [34]bN/AcN/AN/ASuicide attempts, decrease; adaptive attitudes about suicide, increase; knowledge about suicide, increase

Bailey et al [35]N/AN/ADistress, decrease at T1d and T2d; adaptive attitudes about suicide, increase; knowledge about suicide increase; help seeking for self at T2 and T3d, increase

Freedenthal [36]N/AN/AN/AN/AHelp seeking by hotline, increase; help seeking from adult, decrease; help seeking from a peer, decrease

Haas et al [37]N/AN/AN/AN/AOutpatient treatment initiation for students who received evaluation and dialogue with counselor, increase

Han et ala [38]N/AN/AN/AKnowledge of suicide, increase at posttest; attitude about help seeking from a professional, increase

Pisani et ala,e [39]N/AN/AN/ANot reported

Robinson et al [40]N/AN/AN/ANot reported

Schilling et ala [41]N/AN/AN/ASuicidal ideation, planning, or attempts decrease among intervention participants with pretest suicidal ideation; knowledge about suicide, increase

Wyman et ala [42]N/AN/APerceptions of seeking help from adults, increase; help seeking from nonmental health professional, increase; help seeking from a peer, increase
Selective

Dickter et al [43]N/AN/AN/AN/ASuicidal ideation, decrease

Hetrick et al [44]N/AN/AN/AN/ASuicidal ideation, decrease; depression symptoms, significant decrease

Iorfino et al [45]N/AN/AN/AN/AN/ANot reported

King et ala [46]N/AStigma to seek help for mental health issues, decrease; help seeking from a mental health professional, increase; help seeking from family members, increase; help seeking from a peer, increase; outpatient treatment initiation, increase
Indicated

Bertolote et ala [47]N/AN/AN/AN/ANot reported

Czyz et ala [48]N/AN/AN/AN/ACoping for suicidal behavior, increase; coping with safety plan, increase

Hetrick et ala [49]N/AN/AN/ANot reported

King et ala [50]N/AN/AN/AN/ASuicidal ideation, decrease at 6 weeks and 6 months of follow-up among multiple attempters

King et al [51]N/AN/AN/AN/ASuicidal ideation, decrease; distress, decrease

Mehlum et ala [52]N/AN/AN/ASuicidal ideation, decrease; depression symptoms, decrease

Normand et al [53]N/AN/AN/AN/ANot reported

O’Brien et al [54]N/AN/AN/AN/AN/ANot reported

Rosenbaum et ala
[55]
N/AN/AN/AOutpatient treatment initiation, increase

Tan et ala,e [56]N/AN/AN/AN/AN/ANot reported

Yen et al [57]N/AN/AN/AN/ASuicidal ideation, decrease

Yen et al [58]N/AN/AN/AN/ANot reported
All tiers

Silverstone et al [59]N/AN/AN/AN/ASuicidal ideation, planning, and attempts, decrease among actively suicidal participants; depression symptoms, decrease; anxiety, decrease

aIndicates the study was a randomized controlled trial.

bThe study reports that outcome.

cN/A: not applicable.

dT1, T2, and T3 indicate the time trials of the larger study.

eThe study’s purpose was to promote help seeking, but no variables directly reported the help-seeking behaviors of the participants.

In total, 9 of the 12 RCT studies and 8 studies of other designs reported significant changes in study outcomes. A total of 22 studies measured suicidality outcomes and only 9 of those studies reported significant changes (4 of which were RCTs). The next most common outcome domain measured was co-occurring mental health issues, and 5 of the 12 studies reported significant outcomes. In total, 6 of the 9 studies that measured perceptions and knowledge of suicide and 6 of the 8 studies that measured help-seeking behaviors noted significant findings. Finally, only 1 of the 5 studies that measured coping behaviors and 3 of the 8 studies that measured treatment initiation reported significant results.


Principal Findings

The findings suggest that suicidologists around the world are working to utilize technology to prevent youth suicide. Results demonstrate that 17 interventions of varying study designs reported significant changes in at least one of the outcome domains. In addition, it is promising that the majority of selected studies were conducted in educational settings, which may increase opportunities for youth to learn about suicide risk, seek help, and participate in treatment beyond a formal, clinical setting. On the basis of the findings presented in Tables 1-3, these school-based and university-based interventions were mostly universal interventions and aligned with the illness and risk recognition and help-seeking stages of the continuum of care. In addition, these interventions mainly used videos and web-based platforms and predominantly demonstrated efficacy in increasing help-seeking behaviors and youth knowledge and perceptions about suicide.

However, the results from this review also illustrate that efforts are needed to test technology-enhanced interventions across the continuum of care. Among this review’s sample, few studies used technology to assess suicidality or to formally initiate mental health treatment. This gap in the continuum of care is crucial to address as these stages impact participation in treatment modules and, hopefully, prevent future deaths. During the initial search, the investigators noticed that many available electronic assessments were not specific to youth suicidality, had not been incorporated into an intervention trial, or were only in the early stages of development.

Regarding the other intervention tiers, only 4 studies, all using web-based platforms, were determined to be selective interventions and these studies spanned the stages of the continuum of care. Although 3 studies did note significant changes in multiple outcome domains, future research should focus on strategies to use technology to reach youths who are at higher risk of suicide across the continuum of care, as this appears to be an overlooked group in technology-enhanced interventions. It was also not surprising that the majority of indicated interventions addressed the treatment module, treatment engagement, and follow-up stages of care; these interventions also primarily used phones. Although several studies have reported significant improvements in youth suicidal behavior, future trials may extend treatment outcomes beyond suicidality to include coping behaviors (in this review, only 2 studies measured this) and other metrics that mark improvements in youth resiliency.

Although certain tiers of intervention and stages of the continuum of care were associated with specific types of technology, it may be premature to determine whether one modality is better associated with efficacy, acceptability, or feasibility. For example, it would be presumed that studies that use phones would achieve more successful metrics than a study that used a less-established or newer technology, such as a mobile phone app or web-based platform. However, this review suggests that studies with significant findings do not use one specific type of technology and therefore other factors, such as study design, intervention curriculum, and youth sample, may have a greater impact on an intervention’s success.

Implications for Future Study Design

This review emphasizes that not all youth interventions that use technology are the same. Some interventions have more human and face-to-face involvement, whereas others are mainly automated or self-directed by the participant. Behavioral intervention technology (BIT) is a term used in other health disciplines to determine the level of human involvement and automation in an intervention [60]. In contrast, the suicidology literature most often evaluates the intervention as a whole and does not provide specific details about the technological components. Understanding the level of provider integration and the technological components of the BIT may also inform how interventions can be implemented in other settings and scaled to reach a broader youth consumer base [61]. For example, the review’s findings confirm that universal interventions that target help seeking, illnesses, and risk recognition are already tested on a larger scale versus selective and indicated interventions. Therefore, incorporating the BIT terminology and models may help suicidologists determine the efficacy and acceptability of specific intervention components, determine which technologies are better suited for specific stages of the continuum of care, and to disseminate other tiers of interventions to mental health systems that service at-risk and suicidal youth.

Scores on the MQRS demonstrated variability in the quality of the studies and may be a consequence of not restricting the study design for this systematic review. For example, several non-RCT studies did not have multiple sessions or have multisite trials for their intervention. However, many studies, regardless of study design, have underdeveloped MQRS domains in common, such as documenting the study’s theoretical foundation, conducting the study at more than 1 site, and reporting inclusion of a collateral data source. The limited inclusion of collateral data sources is problematic as it is considered best practice to supplement youth self-report with parent, guardian, or teacher observations in face-to-face interventions [62-65]. The issue of an incomplete client profile is most likely indicative of the challenges of data collection and digital interventions [66] and may be an issue that suicidologists implementing technology-enhanced interventions can collectively explore and tackle.

The scores on the MQRS also suggested that few studies adapted interventions to the cultural, social, and linguistic needs of their specific demographic. In addition, no study has reported the participants’ sexual identity, although suicide risk is particularly heightened among sexual minority youth [67]. As global research demonstrates that youth suicidality varies across age cohorts and demographics [5], technology-enhanced interventions need to mirror and be tailored for this diversity.

Implications for the Profession of Suicidology

Diversity among suicide scholars and professional perspectives may also influence the impact that technology-enhanced interventions have on youths’ mental health [68-70]. It is promising that the first authors of 16 studies identified as female, considering the noted gender bias in peer review and grant funding [23,24,71]. This bias has been noted in suicidology as well, as the American Association of Suicidology has historically bestowed more men than women with its annual research awards [72]. In addition, doctoral training in psychology was most common among the first authors. Although this finding is not reflective of the potential professional diversity of the research team and of nonintervention studies, the subfield of technology-enhanced youth interventions may be mindful of how to mentor students and researchers from other disciplines to be principal investigators. Disciplines may include those who are involved in the frontline (such as nursing) or those who have extensive training in digital health literacy and computational methods (ie, health communication fields).

Limitations

As the study sample was not restricted by research design, the investigators could not compare outcome effect sizes. Preliminary searches demonstrated that the pool of eligible studies that were also RCTs would be small and that the investigators did not want to overlook cutting-edge interventions that were in earlier stages of development. In addition, as this review was restricted to specific search guidelines and because suicide research is ever evolving, relevant studies may not have been included. For example, many studies were ineligible as they did not collect or report participants’ ages or validate that the participants were within the specific age range.

Conclusions

This systematic review emphasizes the need for technology-enhanced interventions that extend beyond illness or risk recognition and help seeking, which are developed for diverse youth populations. Although technology shows promise in its utility to address suicidality and increase proactive behaviors, such as help seeking and coping skills, it is difficult to determine which types of technology are better associated with intervention efficacy, acceptability, and feasibility and better suited for specific stages of the continuum of care. The field of suicidology also faces challenges in capturing youth participants’ demographics on digital platforms, supplementing youth self-reports with collateral information, developing interventions suitable for underserved demographics, and involving researchers from diverse backgrounds and disciplines. Adoption of BIT terminology and frameworks may improve the understanding of both the roles of providers and technological components in technology-enhanced suicide preventive interventions for youth and how these interventions can be successfully implemented across the continuum of care and within mental health care systems.

Acknowledgments

Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number T32MH019960. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Literature search terms and adapted study quality scoring sheet.

DOCX File , 26 KB

  1. 10 leading causes of death by age group, United States, 2017. Center for Disease Control and Prevention (CDC). 2018.   URL: https://www.cdc.gov/injury/wisqars/LeadingCauses.html [accessed 2019-12-12]
  2. Suicide. World Health Organization (WHO). 2019.   URL: https://www.who.int/news-room/fact-sheets/detail/suicide [accessed 2019-12-12]
  3. Posner K, Oquendo MA, Gould M, Stanley B, Davies M. Columbia Classification Algorithm of Suicide Assessment (C-CASA): classification of suicidal events in the FDA's pediatric suicidal risk analysis of antidepressants. Am J Psychiatry 2007 Jul;164(7):1035-1043 [FREE Full text] [CrossRef] [Medline]
  4. Nock MK, Green JG, Hwang I, McLaughlin KA, Sampson NA, Zaslavsky AM, et al. Prevalence, correlates, and treatment of lifetime suicidal behavior among adolescents: results from the National Comorbidity Survey Replication Adolescent Supplement. JAMA Psychiatry 2013 Mar;70(3):300-310 [FREE Full text] [CrossRef] [Medline]
  5. McLoughlin AB, Gould MS, Malone KM. Global trends in teenage suicide: 2003-2014. QJM 2015 Oct 31;108(10):765-780. [CrossRef] [Medline]
  6. Behar L. Changing Patterns of State Responsibility: A Case Study of North Carolina. Journal of Clinical Child Psychology 2010 Jun 07;14(3):188-195. [CrossRef]
  7. Stroul B, Friedman R. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center; 1986.
  8. Gould MS, Lake AM, Galfalvy H, Kleinman M, Munfakh JL, Wright J, et al. Follow-up with Callers to the National Suicide Prevention Lifeline: Evaluation of Callers' Perceptions of Care. Suicide Life Threat Behav 2018 Feb;48(1):75-86. [CrossRef] [Medline]
  9. MacDonald K, Fainman-Adelman N, Anderson KK, Iyer SN. Pathways to mental health services for young people: a systematic review. Soc Psychiatry Psychiatr Epidemiol 2018 Oct;53(10):1005-1038 [FREE Full text] [CrossRef] [Medline]
  10. Berrouiguet S, Courtet P, Larsen M, Walter M, Vaiva G. Suicide prevention: Towards integrative, innovative and individualized brief contact interventions. Eur Psychiatry 2018 Jan;47:25-26. [CrossRef] [Medline]
  11. Anderson M, Jiang, J. 2018.   URL: https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/ [accessed 2019-12-12]
  12. Smartphone ownership is growing rapidly around the world, but not always equally. Taylor K, Silver L. 2019.   URL: https:/​/www.​pewresearch.org/​global/​2019/​02/​05/​smartphone-ownership-is-growing-rapidly-around-the-world-but-not-always-equally/​ [accessed 2019-12-12]
  13. Kreuze E, Jenkins C, Gregoski M, York J, Mueller M, Lamis DA, et al. Technology-enhanced suicide prevention interventions: A systematic review. J Telemed Telecare 2017 Jul;23(6):605-617. [CrossRef] [Medline]
  14. Kreuze E, Ruggiero KJ. Technology-Oriented Suicide Prevention Interventions for Adolescents and Adolescent Gatekeepers: A Qualitative Review. Adolescent Res Rev 2017 Jun 6;3(2):219-233. [CrossRef]
  15. Christensen H, Batterham P, O'Dea B. E-health interventions for suicide prevention. Int J Environ Res Public Health 2014 Aug 12;11(8):8193-8212 [FREE Full text] [CrossRef] [Medline]
  16. Reyes-Portillo JA, Mufson L, Greenhill LL, Gould MS, Fisher PW, Tarlow N, et al. Web-based interventions for youth internalizing problems: a systematic review. J Am Acad Child Adolesc Psychiatry 2014 Dec;53(12):1254-1270.e5. [CrossRef] [Medline]
  17. Perry Y, Werner-Seidler A, Calear AL, Christensen H. Web-Based and Mobile Suicide Prevention Interventions for Young People: A Systematic Review. J Can Acad Child Adolesc Psychiatry 2016;25(2):73-79 [FREE Full text] [Medline]
  18. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
  19. Young teens (ages 12-14 years of age). Center for Disease Control and Prevention(CDC). 2019.   URL: https://www.cdc.gov/ncbddd/childdevelopment/positiveparenting/adolescence.html [accessed 2019-12-12]
  20. World Health Organization (WHO). Adolescent health in the South-East Asia region. 2020.   URL: https://www.who.int/southeastasia/health-topics/adolescent-health [accessed 2019-12-12]
  21. Endnote. Citation and reference software. Philadelphia, PA: Clarivate Analytics; 2020.
  22. Covidence. Systematic review software. Melboure: Veritas Health Innovation; 2020.
  23. Gender in the Global Research Landscape. Elsevier. 2017.   URL: https://www.elsevier.com/research-intelligence/campaigns/gender-17 [accessed 2019-12-12]
  24. Ross E. Gender bias distorts peer review across fields. Nature News. 2017 Mar 23.   URL: https://www.nature.com/news/gender-bias-distorts-peer-review-across-fields-1.21685 [accessed 2020-05-01]
  25. Gabert R, Ng M, Sogarwal R, Bryant M, Deepu RV, McNellan CR, et al. Identifying gaps in the continuum of care for hypertension and diabetes in two Indian communities. BMC Health Serv Res 2017 Dec 27;17(1):846 [FREE Full text] [CrossRef] [Medline]
  26. Graboyes EM, Kompelli AR, Neskey DM, Brennan E, Nguyen S, Sterba KR, et al. Association of Treatment Delays With Survival for Patients With Head and Neck Cancer: A Systematic Review. JAMA Otolaryngol Head Neck Surg 2019 Feb 01;145(2):166-177 [FREE Full text] [CrossRef] [Medline]
  27. Tofighi B, Sindhu SS, Chemi C, Lewis CF, Dickson VV, Lee JD. Perspectives on the HIV continuum of care among adult opioid users in New York City: a qualitative study. Harm Reduct J 2019 Oct 12;16(1):58 [FREE Full text] [CrossRef] [Medline]
  28. Miller W, Brown J, Simpson T. What works? A methodological analysis of the alcohol treatment outcome literature. Needham Heights, MA: Allyn & Bacon; 1995:12-44.
  29. Miller W, Wilbourne PL. Mesa Grande: a methodological analysis of clinical trials of treatments for alcohol use disorders. Addiction 2002 Mar;97(3):265-277. [CrossRef] [Medline]
  30. Whiteman KL, Naslund JA, DiNapoli EA, Bruce ML, Bartels SJ. Systematic Review of Integrated General Medical and Psychiatric Self-Management Interventions for Adults With Serious Mental Illness. Psychiatr Serv 2016 Nov 01;67(11):1213-1225 [FREE Full text] [CrossRef] [Medline]
  31. Vaughn MG, Howard MO. Integrated psychosocial and opioid-antagonist treatment for alcohol dependence: A systematic review of controlled evaluations. Social Work Research 2004 Mar 01;28(1):41-53. [CrossRef]
  32. Cabassa LJ, Camacho D, Vélez-Grau CM, Stefancic A. Peer-based health interventions for people with serious mental illness: A systematic literature review. J Psychiatr Res 2017 Jan;84:80-89. [CrossRef] [Medline]
  33. Auslander W, Fisher C, Ollie M, Yu M. Teaching Master's and Doctoral Social Work Students to Systematically Evaluate Evidence-Based Interventions. Journal of Teaching in Social Work 2012 Sep;32(4):320-341. [CrossRef]
  34. Aseltine RH, DeMartino R. An outcome evaluation of the SOS Suicide Prevention Program. Am J Public Health 2004 Mar;94(3):446-451. [CrossRef] [Medline]
  35. Bailey E, Krysinska K, O'Dea B, Robinson J. Internet Forums for Suicide Bereavement. Crisis 2017 Nov;38(6):393-402. [CrossRef] [Medline]
  36. Freedenthal S. Adolescent help-seeking and the Yellow Ribbon Suicide Prevention Program: an evaluation. Suicide Life Threat Behav 2010 Dec;40(6):628-639. [CrossRef] [Medline]
  37. Haas A, Koestner B, Rosenberg J, Moore D, Garlow SJ, Sedway J, et al. An Interactive Web-Based Method of Outreach to College Students at Risk for Suicide. Journal of American College Health 2008 Jul;57(1):15-22. [CrossRef]
  38. Han J, Batterham PJ, Calear AL, Wu Y, Xue J, van Spijker BAJ. Development and pilot evaluation of an online psychoeducational program for suicide prevention among university students: A randomised controlled trial. Internet Interv 2018 Jun;12:111-120 [FREE Full text] [CrossRef] [Medline]
  39. Pisani AR, Wyman PA, Gurditta K, Schmeelk-Cone K, Anderson CL, Judd E. Mobile Phone Intervention to Reduce Youth Suicide in Rural Communities: Field Test. JMIR Ment Health 2018 May 31;5(2):e10425 [FREE Full text] [CrossRef] [Medline]
  40. Robinson J, Bailey E, Hetrick S, Paix S, O'Donnell M, Cox G, et al. Developing Social Media-Based Suicide Prevention Messages in Partnership With Young People: Exploratory Study. JMIR Ment Health 2017 Oct 04;4(4):e40 [FREE Full text] [CrossRef] [Medline]
  41. Schilling EA, Lawless M, Buchanan L, Aseltine RH. "Signs of Suicide" shows promise as a middle school suicide prevention program. Suicide Life Threat Behav 2014 Dec;44(6):653-667. [CrossRef] [Medline]
  42. Wyman PA, Brown CH, LoMurray M, Schmeelk-Cone K, Petrova M, Yu Q, et al. An outcome evaluation of the Sources of Strength suicide prevention program delivered by adolescent peer leaders in high schools. Am J Public Health 2010 Sep;100(9):1653-1661. [CrossRef] [Medline]
  43. Dickter B, Bunge EL, Brown LM, Leykin Y, Soares EE, Van Voorhees B, et al. Impact of an online depression prevention intervention on suicide risk factors for adolescents and young adults. Mhealth 2019;5:11 [FREE Full text] [CrossRef] [Medline]
  44. Hetrick SE, Goodall J, Yuen HP, Davey CG, Parker AG, Robinson J, et al. Comprehensive Online Self-Monitoring to Support Clinicians Manage Risk of Suicide in Youth Depression. Crisis 2017 May;38(3):147-157. [CrossRef] [Medline]
  45. Iorfino F, Davenport TA, Ospina-Pinillos L, Hermens DF, Cross S, Burns J, et al. Using New and Emerging Technologies to Identify and Respond to Suicidality Among Help-Seeking Young People: A Cross-Sectional Study. J Med Internet Res 2017 Jul 12;19(7):e247 [FREE Full text] [CrossRef] [Medline]
  46. King CA, Eisenberg D, Zheng K, Czyz E, Kramer A, Horwitz A, et al. Online suicide risk screening and intervention with college students: a pilot randomized controlled trial. J Consult Clin Psychol 2015 Jun;83(3):630-636 [FREE Full text] [CrossRef] [Medline]
  47. Bertolote JM, Fleischmann A, De Leo D, Phillips MR, Botega NJ, Vijayakumar L, et al. Repetition of suicide attempts: data from emergency care settings in five culturally different low- and middle-income countries participating in the WHO SUPRE-MISS Study. Crisis 2010;31(4):194-201. [CrossRef] [Medline]
  48. Czyz EK, King CA, Biermann BJ. Motivational Interviewing-Enhanced Safety Planning for Adolescents at High Suicide Risk: A Pilot Randomized Controlled Trial. J Clin Child Adolesc Psychol 2019;48(2):250-262. [CrossRef] [Medline]
  49. Hetrick SE, Yuen HP, Bailey E, Cox GR, Templer K, Rice SM, et al. Internet-based cognitive behavioural therapy for young people with suicide-related behaviour (Reframe-IT): a randomised controlled trial. Evid Based Ment Health 2017 Aug;20(3):76-82. [CrossRef] [Medline]
  50. King CA, Klaus N, Kramer A, Venkataraman S, Quinlan P, Gillespie B. The Youth-Nominated Support Team–Version II for suicidal adolescents: A randomized controlled intervention trial. Journal of Consulting and Clinical Psychology 2009 Oct;77(5):880-893 [FREE Full text] [CrossRef] [Medline]
  51. King R, Nurcombe B, Bickman L, Hides L, Reid W. Telephone counselling for adolescent suicide prevention: changes in suicidality and mental state from beginning to end of a counselling session. Suicide Life Threat Behav 2003;33(4):400-411. [CrossRef] [Medline]
  52. Mehlum L, Tørmoen AJ, Ramberg M, Haga E, Diep LM, Laberg S, et al. Dialectical behavior therapy for adolescents with repeated suicidal and self-harming behavior: a randomized trial. J Am Acad Child Adolesc Psychiatry 2014 Oct;53(10):1082-1091. [CrossRef] [Medline]
  53. Normand D, Colin S, Gaboulaud V, Baubet T, Taieb O. How to stay in touch with adolescents and young adults after a suicide attempt? Implementation of a 4-phones-calls procedure over 1 year after discharge from hospital, in a Parisian suburb. Encephale 2018 Sep;44(4):301-307. [CrossRef] [Medline]
  54. McManama O'Brien KH, LeCloux M, Ross A, Gironda C, Wharff EA. A Pilot Study of the Acceptability and Usability of a Smartphone Application Intervention for Suicidal Adolescents and Their Parents. Arch Suicide Res 2017;21(2):254-264. [CrossRef] [Medline]
  55. Asarnow JR, Baraff LJ, Berk M, Grob CS, Devich-Navarro M, Suddath R, et al. An emergency department intervention for linking pediatric suicidal patients to follow-up mental health treatment. Psychiatr Serv 2011 Nov;62(11):1303-1309 [FREE Full text] [CrossRef] [Medline]
  56. Tan Z, Liu X, Liu X, Cheng Q, Zhu T. Designing Microblog Direct Messages to Engage Social Media Users With Suicide Ideation: Interview and Survey Study on Weibo. J Med Internet Res 2017 Dec 12;19(12):e381 [FREE Full text] [CrossRef] [Medline]
  57. Yen S, Ranney ML, Tezanos KM, Chuong A, Kahler CW, Solomon JB, et al. Skills to Enhance Positivity in Suicidal Adolescents: Results From an Open Development Trial. Behav Modif 2019 Mar;43(2):202-221 [FREE Full text] [CrossRef] [Medline]
  58. Yen S, Spirito A, Weinstock L, Tezanos K, Kolobaric A, Miller I. Coping long term with active suicide in adolescents: Results from a pilot randomized controlled trial. Clin Child Psychol Psychiatry 2019;24(4):847-859. [CrossRef] [Medline]
  59. Silverstone PH, Bercov M, Suen VYM, Allen A, Cribben I, Goodrick J, et al. Long-term Results from the Empowering a Multimodal Pathway Toward Healthy Youth Program, a Multimodal School-Based Approach, Show Marked Reductions in Suicidality, Depression, and Anxiety in 6,227 Students in Grades 6-12 (Aged 11-18). Front Psychiatry 2017;8:81 [FREE Full text] [CrossRef] [Medline]
  60. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the Trial: Systematic Review of Real-World Uptake and Engagement With Digital Self-Help Interventions for Depression, Low Mood, or Anxiety. J Med Internet Res 2018 Jun 06;20(6):e199 [FREE Full text] [CrossRef] [Medline]
  61. Hermes ED, Lyon AR, Schueller SM, Glass JE. Measuring the Implementation of Behavioral Intervention Technologies: Recharacterization of Established Outcomes. J Med Internet Res 2019 Jan 25;21(1):e11752 [FREE Full text] [CrossRef] [Medline]
  62. Ash P. Children Are Different: Liability Issues in Working With Suicidal and Dangerous Youths. Focus (Am Psychiatr Publ) 2019 Oct;17(4):355-359. [CrossRef] [Medline]
  63. Thompson EC, Fox KA, Lapomardo A, Hunt JI, Wolff JC. Youth Self Report Thought Problems and Sleep Difficulties Are Linked to Suicidal Ideation Among Psychiatrically Hospitalized Adolescents. J Child Adolesc Psychopharmacol 2020 Feb 13. [CrossRef] [Medline]
  64. Van Meter AR, Algorta GP, Youngstrom EA, Lechtman Y, Youngstrom JK, Feeny NC, et al. Assessing for suicidal behavior in youth using the Achenbach System of Empirically Based Assessment. Eur Child Adolesc Psychiatry 2018 Feb;27(2):159-169 [FREE Full text] [CrossRef] [Medline]
  65. Van Roy B, Groholt B, Heyerdahl S, Clench-Aas J. Understanding discrepancies in parent-child reporting of emotional and behavioural problems: Effects of relational and socio-demographic factors. BMC Psychiatry 2010 Jul 16;10(1). [CrossRef]
  66. Berman NC, Sullivan A, Wilhelm S, Cohen IG. Effect of a legal prime on clinician's assessment of suicide risk. Death Stud 2016;40(1):61-67. [CrossRef] [Medline]
  67. Center for Disease Prevention and Control (CDC). 2018.   URL: https://www.cdc.gov/healthyyouth/data/yrbs/pdf/trendsreport.pdf [accessed 2020-12-12]
  68. Abrutyn S, Mueller AS. Toward a robust science of suicide: Epistemological, theoretical, and methodological considerations in advancing suicidology. Death Stud 2019 Sep 10:1-6. [CrossRef] [Medline]
  69. Maple M, Pearce T, Sanford RL, Cerel J. The Role of Social Work in Suicide Prevention, Intervention, and Postvention: A Scoping Review. Australian Social Work 2016 Aug 31;70(3):289-301. [CrossRef]
  70. Standley CJ. Expanding our paradigms: Intersectional and socioecological approaches to suicide prevention. Death Stud 2020 Feb 12:1-9. [CrossRef] [Medline]
  71. Witteman HO, Hendricks M, Straus S, Tannenbaum C. Are gender gaps due to evaluations of the applicant or the science? A natural experiment at a national funding agency. The Lancet 2019 Feb;393(10171):531-540. [CrossRef]
  72. Canetto S. Suicide as culturally scripted: Theory, evidence, and implications for prevention. In: Annual American Association Suicidology Conference. 2020 Presented at: American Association Suicidology; April 2020; Virtual conference.


BIT: behavioral intervention technology
MQRS: Methodological Quality Rating Scale
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
RCT: randomized controlled trial


Edited by G Eysenbach; submitted 11.03.20; peer-reviewed by Y Perry, P Batterham, K McKay; comments to author 20.04.20; revised version received 26.05.20; accepted 25.06.20; published 09.10.20

Copyright

©Hannah Szlyk, Jia Tan. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.10.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.