Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 09.10.20 in Vol 22, No 10 (2020): October

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/18672, first published Mar 11, 2020.

This paper is in the following e-collection/theme issue:

    Review

    The Role of Technology and the Continuum of Care for Youth Suicidality: Systematic Review

    1School of Social Work, Rutgers, The State University of New Jersey, New Brunswick, NJ, United States

    2The Brown School, Washington University in St Louis, Saint Louis, MO, United States

    *all authors contributed equally

    Corresponding Author:

    Hannah Szlyk, MSW, PhD

    School of Social Work

    Rutgers, The State University of New Jersey

    390 George Street

    New Brunswick, NJ, 08901

    United States

    Phone: 1 9178364225

    Email: hannah.szlyk@rutgers.edu


    ABSTRACT

    Background: Youth suicide is a global public health issue, and using technology is one strategy to increase participation in preventive interventions. However, there is minimal knowledge on how technology-enhanced interventions for youth correspond to the stages of care, from illness or risk recognition to treatment follow-up.

    Objective: This systematic review aims to examine the efficacy of technology-enhanced youth suicide prevention and interventions across the continuum of care.

    Methods: Four electronic databases were searched up to spring 2019 for youth suicide preventive interventions that used technology. The review was not restricted by study design and eligible studies could report outcomes on suicidality or related behaviors, such as formal treatment initiation. An adapted version of the Methodological Quality Ratings Scale was used to assess study quality.

    Results: A total of 26 studies were identified. The findings support the emerging efficacy of technology-enhanced interventions, including a decline in suicidality and an increase in proactive behaviors. However, evidence suggests that there are gaps in the continuum of care and recent study samples do not represent the diverse identities of vulnerable youth.

    Conclusions: The majority of identified studies were conducted in school settings and were universal interventions that aligned with the illness and risk recognition and help-seeking stages of the continuum of care. This field could be strengthened by having future studies target the stages of assessment and treatment initiation, include diverse youth demographics, and examine the varying roles of providers and technological components in emerging interventions.

    J Med Internet Res 2020;22(10):e18672

    doi:10.2196/18672

    KEYWORDS



    Introduction

    Background

    Youth suicide is a global public health crisis. In the United States, suicide is the second leading cause of death for children and youth aged 10-24 years [1]. Globally, suicide is the third leading cause of death for youths aged 15-19 years [2]. In addition to deaths by suicide, suicidality includes suicidal ideation and related behaviors, such as plans to attempt suicide and actual suicide attempts [3]. Thus, research suggests that the risk of youth suicide may even be more pervasive as many youths experience suicidal ideation and nonfatal suicidal behaviors [4]. Universally, adolescence and early adulthood are vulnerable periods for when suicide risk is particularly elevated [5].

    Therefore, it is important that youths have access to global systems of mental health care. Mental health services and resources may range from promoting illness recognition (in the case of suicidality, this may include risk factor recognition) to providing targeted treatment and to offering follow-up services. This range, which spans the levels of intensity in care, is often called the continuum of care [6,7]. Suicidal individuals who are engaged in an integrated continuum of mental health care may experience decreases in suicidality [8]. However, youth engagement in the continuum of mental health care is often complicated as a consequence of developmental changes, the delayed detection of symptoms, and delayed access to treatment [9]. Thus, researchers need to ensure that available interventions are tailored to specifically meet youths’ needs and correspond with the stages of the continuum of care [9].

    Technology is one of the identified mediums to bridge gaps in the continuum of suicide preventive interventions [10]. Technology is especially relevant to engaging youths around the world. Research suggests that most young people in the United States and in developed countries have access to smartphones [11,12], whereas access increases among younger cohorts of emerging economies [12]. Therefore, the use of technology may address barriers to face-to-face care, such as access, reach, and stigma [10,13]. Technology-enhanced interventions use technology to solely deliver or serve as a component of an intervention and can include a mobile phone app, text messaging, telephone, videos, and web-based platforms [13]. Previous reviews on this topic have been restricted to gatekeeper interventions [14], including interventions across the lifespan or interventions designed to address broad mental health issues [15], focused on specific technologies [16], or that may be outdated as new interventions have since been developed [17]. No known review has explored how current technology-enhanced suicide interventions for youth correspond to the stages of the continuum of care. Thus, there is an incomplete understanding of the breadth and efficacy of preventive interventions that use technology and serve youth at risk of suicide.

    Objectives

    To address these gaps in the literature, this systematic review aims to examine the efficacy of technology-enhanced youth suicide prevention and interventions across the continuum of care. The authors evaluated study outcomes in addition to suicidality, including help-seeking behaviors and coping skills, to better assess how the literature supports youth in leading lives worth living. The findings have implications for how suicidology may address identified gaps in the stages of the continuum of technology-enhanced suicide interventions and enhance care for vulnerable youths.


    Methods

    Search Strategy

    The search was conducted in spring 2019, and the systematic review adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (Figure 1) [18].

    Electronic databases were searched (PsycINFO, CINAHL [Cumulated Index to Nursing and Allied Health Literature], Ovid MEDLINE, ClinicalTrials.gov) using search terms specific to youth, suicidality, technology, and interventions (Multimedia Appendix 1). The study selection and data extraction were conducted by 2 assessors (the first and second authors). Eligible studies were required to focus on youths who were at potential risk, at high risk, or struggling with suicidality, be in English, and have the majority of participants between the ages of 12 and 24 years [19,20]. Selected studies adhered to the definition of technology-enhanced interventions (as previously defined) by Kreuze et al [13]. The investigators decided not to exclude studies that had primary outcome variables other than suicidality. This decision was based on a preliminary search of the literature, in which the investigators noticed that studies of technology-enhanced interventions for youth fell into several primary outcome domains (see the Data Extraction section for the outcome domains explored). The intention was to identify studies that may have been overlooked in previous reviews restricted to outcomes of suicidality and that applied to the stages of the continuum of care. Relevant systematic reviews were also cross-checked to adjust initial search terms and to potentially identify studies that had been missed in the final search.

    Studies that focused on assisted suicide, nonsuicidal self-injury, postvention, or only gatekeeper outcomes were excluded from the final sample. In addition, the investigators excluded studies that only used technology to collect information about participant characteristics and behaviors, which only provided qualitative results, or only discussed the psychometrics of their assessment tool. No exclusion was placed on the trial design. When multiple publications of the same intervention were identified, the most recent or the most advanced trial was selected. Preliminary searches were organized using Endnote [21], and screening and data extraction were conducted using Covidence [22] and spreadsheets.

    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram for search strategy.
    View this figure

    Data Extraction

    Data were extracted for study characteristics (Tables 1 and 2). As gender bias in peer review and in first author publications exists [23,24], the first author’s gender identity was extracted and determined by searching department profiles. In addition, as suicidology is considered a multidisciplinary field, the investigators decided to document the first author’s field of degree by searching web-based profiles and curriculum vitae. Studies’ stages on the continuum of care were also extracted, and studies could cover more than 1 stage. The continuum of care framework used in this review was based on the literature from various health disciplines [25-27] and included the following stages: illness or suicide risk factor recognition, help seeking, assessment, treatment initiation, treatment module, treatment engagement, and follow-up.

    Quality of study design and methodology were assessed using an adapted version of the Methodological Quality Rating Scale (MQRS; see Multimedia Appendix 1 for full scale) [28]. The MQRS was originally developed to review alcohol treatment outcomes and covered 12 domains, such as study design, documentation of quality control of treatments, and reports of participants lost to attrition [28,29]. Each domain is rated by the study’s strategies to decrease bias, with studies reporting more rigorous strategies receiving a higher quality score. The MQRS has been used to review other substance and mental health intervention studies [30,31] and has been adapted to evaluate the inclusion of theoretical frameworks and cultural linguistic adaptions in study development [32,33]. The investigators assessed interrater reliability by calculating the percent agreement for each MQRS domain. For example, there was an 81% agreement on both investigators’ ratings by the domain of study design (see Multimedia Appendix 1 for the percent agreement for all MQRS domains).

    The outcome attainment of each selected study was evaluated using a categorization system for statistical significance and outcome direction [33]. Outcomes were grouped into the following domains: suicidality (ie, suicidal ideation, planning suicide attempts, and attempts), co-occurring mental health issues (ie, anxiety, depression), youth’s perceptions and knowledge about suicide, help-seeking behaviors, proactive coping behaviors, and formal treatment initiation, which is the official beginning of participation in outpatient or inpatient mental health treatment.

    The investigators documented whether the outcomes related to these domains demonstrated statistically significant changes (at least P<.05) and whether the change occurred in the desired direction (ie, a decrease in suicidal ideation) [33]. Differences in coding and ratings during the phases of study screening and data extraction were resolved through consensus between the 2 investigators.


    Results

    Study Characteristics

    The final sample included 26 studies (Table 1) [34-59]. Although various professional disciplines were represented (ie, social work, medicine, public health), over half (n=16) of the first authors held a doctoral degree in psychology. In total, 16 of the first authors were female; 13 studies were conducted in the United States. Among studies that provided participant ages, the mean or median age ranged from 14.7 to 23 years. Female participants represented the majority of the sample participants. Of the studies that did report youth ethnicity, the majority of participants identified as White; no study reported the sexual or gendered minority identity of the participants. A total of 13 studies were conducted in an educational setting.

    The selected studies included a variety of preventive interventions (Table 2); the SOS (Signs of Suicide) model was included in 2 studies: the original model for high schools [34] and the adapted model for middle schools [41]. The modalities of technology most often used were phones and web or web-based platforms. Half of the studies (n=13) described indicated interventions for youth suicidality. A total of 12 studies were randomized controlled trials (RCTs). The outcomes of the adapted MQRS tool ranged from 0 (lowest quality) to 18 (highest quality), with an average score of 10.4 and range of 3 to 16.

    Table 1. Study characteristics.
    View this table
    Table 2. Study characteristics continued.
    View this table

    Regarding the continuum of care (Table 3), most studies were targeted to increase illness or risk factor recognition (n=11), to increase help seeking (n=10), and to guide youths through a treatment module (n=10; Table 2). The majority of studies (n=18) addressed >1 stage of the continuum of care. For example, 8 of the 9 universal interventions addressed illness or risk recognition and help seeking; 5 of the 11 indicated interventions focused on the stages (at least) of participating in the treatment module and treatment engagement. As illustrated in Table 4, among the 6 common outcome domains measured, most studies (n=22) reported suicidality (ie, ideation, attempts) as an important study variable, followed by co-occurring mental health issues (n=12; ie, depression, distress, or anxiety).

    Table 3. Studies and their stages on the continuum of care.
    View this table
    Table 4. Measured and significant intervention outcomes.
    View this table

    In total, 9 of the 12 RCT studies and 8 studies of other designs reported significant changes in study outcomes. A total of 22 studies measured suicidality outcomes and only 9 of those studies reported significant changes (4 of which were RCTs). The next most common outcome domain measured was co-occurring mental health issues, and 5 of the 12 studies reported significant outcomes. In total, 6 of the 9 studies that measured perceptions and knowledge of suicide and 6 of the 8 studies that measured help-seeking behaviors noted significant findings. Finally, only 1 of the 5 studies that measured coping behaviors and 3 of the 8 studies that measured treatment initiation reported significant results.


    Discussion

    Principal Findings

    The findings suggest that suicidologists around the world are working to utilize technology to prevent youth suicide. Results demonstrate that 17 interventions of varying study designs reported significant changes in at least one of the outcome domains. In addition, it is promising that the majority of selected studies were conducted in educational settings, which may increase opportunities for youth to learn about suicide risk, seek help, and participate in treatment beyond a formal, clinical setting. On the basis of the findings presented in Tables 1-3, these school-based and university-based interventions were mostly universal interventions and aligned with the illness and risk recognition and help-seeking stages of the continuum of care. In addition, these interventions mainly used videos and web-based platforms and predominantly demonstrated efficacy in increasing help-seeking behaviors and youth knowledge and perceptions about suicide.

    However, the results from this review also illustrate that efforts are needed to test technology-enhanced interventions across the continuum of care. Among this review’s sample, few studies used technology to assess suicidality or to formally initiate mental health treatment. This gap in the continuum of care is crucial to address as these stages impact participation in treatment modules and, hopefully, prevent future deaths. During the initial search, the investigators noticed that many available electronic assessments were not specific to youth suicidality, had not been incorporated into an intervention trial, or were only in the early stages of development.

    Regarding the other intervention tiers, only 4 studies, all using web-based platforms, were determined to be selective interventions and these studies spanned the stages of the continuum of care. Although 3 studies did note significant changes in multiple outcome domains, future research should focus on strategies to use technology to reach youths who are at higher risk of suicide across the continuum of care, as this appears to be an overlooked group in technology-enhanced interventions. It was also not surprising that the majority of indicated interventions addressed the treatment module, treatment engagement, and follow-up stages of care; these interventions also primarily used phones. Although several studies have reported significant improvements in youth suicidal behavior, future trials may extend treatment outcomes beyond suicidality to include coping behaviors (in this review, only 2 studies measured this) and other metrics that mark improvements in youth resiliency.

    Although certain tiers of intervention and stages of the continuum of care were associated with specific types of technology, it may be premature to determine whether one modality is better associated with efficacy, acceptability, or feasibility. For example, it would be presumed that studies that use phones would achieve more successful metrics than a study that used a less-established or newer technology, such as a mobile phone app or web-based platform. However, this review suggests that studies with significant findings do not use one specific type of technology and therefore other factors, such as study design, intervention curriculum, and youth sample, may have a greater impact on an intervention’s success.

    Implications for Future Study Design

    This review emphasizes that not all youth interventions that use technology are the same. Some interventions have more human and face-to-face involvement, whereas others are mainly automated or self-directed by the participant. Behavioral intervention technology (BIT) is a term used in other health disciplines to determine the level of human involvement and automation in an intervention [60]. In contrast, the suicidology literature most often evaluates the intervention as a whole and does not provide specific details about the technological components. Understanding the level of provider integration and the technological components of the BIT may also inform how interventions can be implemented in other settings and scaled to reach a broader youth consumer base [61]. For example, the review’s findings confirm that universal interventions that target help seeking, illnesses, and risk recognition are already tested on a larger scale versus selective and indicated interventions. Therefore, incorporating the BIT terminology and models may help suicidologists determine the efficacy and acceptability of specific intervention components, determine which technologies are better suited for specific stages of the continuum of care, and to disseminate other tiers of interventions to mental health systems that service at-risk and suicidal youth.

    Scores on the MQRS demonstrated variability in the quality of the studies and may be a consequence of not restricting the study design for this systematic review. For example, several non-RCT studies did not have multiple sessions or have multisite trials for their intervention. However, many studies, regardless of study design, have underdeveloped MQRS domains in common, such as documenting the study’s theoretical foundation, conducting the study at more than 1 site, and reporting inclusion of a collateral data source. The limited inclusion of collateral data sources is problematic as it is considered best practice to supplement youth self-report with parent, guardian, or teacher observations in face-to-face interventions [62-65]. The issue of an incomplete client profile is most likely indicative of the challenges of data collection and digital interventions [66] and may be an issue that suicidologists implementing technology-enhanced interventions can collectively explore and tackle.

    The scores on the MQRS also suggested that few studies adapted interventions to the cultural, social, and linguistic needs of their specific demographic. In addition, no study has reported the participants’ sexual identity, although suicide risk is particularly heightened among sexual minority youth [67]. As global research demonstrates that youth suicidality varies across age cohorts and demographics [5], technology-enhanced interventions need to mirror and be tailored for this diversity.

    Implications for the Profession of Suicidology

    Diversity among suicide scholars and professional perspectives may also influence the impact that technology-enhanced interventions have on youths’ mental health [68-70]. It is promising that the first authors of 16 studies identified as female, considering the noted gender bias in peer review and grant funding [23,24,71]. This bias has been noted in suicidology as well, as the American Association of Suicidology has historically bestowed more men than women with its annual research awards [72]. In addition, doctoral training in psychology was most common among the first authors. Although this finding is not reflective of the potential professional diversity of the research team and of nonintervention studies, the subfield of technology-enhanced youth interventions may be mindful of how to mentor students and researchers from other disciplines to be principal investigators. Disciplines may include those who are involved in the frontline (such as nursing) or those who have extensive training in digital health literacy and computational methods (ie, health communication fields).

    Limitations

    As the study sample was not restricted by research design, the investigators could not compare outcome effect sizes. Preliminary searches demonstrated that the pool of eligible studies that were also RCTs would be small and that the investigators did not want to overlook cutting-edge interventions that were in earlier stages of development. In addition, as this review was restricted to specific search guidelines and because suicide research is ever evolving, relevant studies may not have been included. For example, many studies were ineligible as they did not collect or report participants’ ages or validate that the participants were within the specific age range.

    Conclusions

    This systematic review emphasizes the need for technology-enhanced interventions that extend beyond illness or risk recognition and help seeking, which are developed for diverse youth populations. Although technology shows promise in its utility to address suicidality and increase proactive behaviors, such as help seeking and coping skills, it is difficult to determine which types of technology are better associated with intervention efficacy, acceptability, and feasibility and better suited for specific stages of the continuum of care. The field of suicidology also faces challenges in capturing youth participants’ demographics on digital platforms, supplementing youth self-reports with collateral information, developing interventions suitable for underserved demographics, and involving researchers from diverse backgrounds and disciplines. Adoption of BIT terminology and frameworks may improve the understanding of both the roles of providers and technological components in technology-enhanced suicide preventive interventions for youth and how these interventions can be successfully implemented across the continuum of care and within mental health care systems.

    Acknowledgments

    Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number T32MH019960. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Literature search terms and adapted study quality scoring sheet.

    DOCX File , 26 KB

    References

    1. 10 leading causes of death by age group, United States, 2017. Center for Disease Control and Prevention (CDC). 2018.   URL: https://www.cdc.gov/injury/wisqars/LeadingCauses.html [accessed 2019-12-12]
    2. Suicide. World Health Organization (WHO). 2019.   URL: https://www.who.int/news-room/fact-sheets/detail/suicide [accessed 2019-12-12]
    3. Posner K, Oquendo MA, Gould M, Stanley B, Davies M. Columbia Classification Algorithm of Suicide Assessment (C-CASA): classification of suicidal events in the FDA's pediatric suicidal risk analysis of antidepressants. Am J Psychiatry 2007 Jul;164(7):1035-1043 [FREE Full text] [CrossRef] [Medline]
    4. Nock MK, Green JG, Hwang I, McLaughlin KA, Sampson NA, Zaslavsky AM, et al. Prevalence, correlates, and treatment of lifetime suicidal behavior among adolescents: results from the National Comorbidity Survey Replication Adolescent Supplement. JAMA Psychiatry 2013 Mar;70(3):300-310 [FREE Full text] [CrossRef] [Medline]
    5. McLoughlin AB, Gould MS, Malone KM. Global trends in teenage suicide: 2003-2014. QJM 2015 Oct 31;108(10):765-780. [CrossRef] [Medline]
    6. Behar L. Changing Patterns of State Responsibility: A Case Study of North Carolina. Journal of Clinical Child Psychology 2010 Jun 07;14(3):188-195. [CrossRef]
    7. Stroul B, Friedman R. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center; 1986.
    8. Gould MS, Lake AM, Galfalvy H, Kleinman M, Munfakh JL, Wright J, et al. Follow-up with Callers to the National Suicide Prevention Lifeline: Evaluation of Callers' Perceptions of Care. Suicide Life Threat Behav 2018 Feb;48(1):75-86. [CrossRef] [Medline]
    9. MacDonald K, Fainman-Adelman N, Anderson KK, Iyer SN. Pathways to mental health services for young people: a systematic review. Soc Psychiatry Psychiatr Epidemiol 2018 Oct;53(10):1005-1038 [FREE Full text] [CrossRef] [Medline]
    10. Berrouiguet S, Courtet P, Larsen M, Walter M, Vaiva G. Suicide prevention: Towards integrative, innovative and individualized brief contact interventions. Eur Psychiatry 2018 Jan;47:25-26. [CrossRef] [Medline]
    11. Anderson M, Jiang, J. 2018.   URL: https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/ [accessed 2019-12-12]
    12. Smartphone ownership is growing rapidly around the world, but not always equally. Taylor K, Silver L. 2019.   URL: https:/​/www.​pewresearch.org/​global/​2019/​02/​05/​smartphone-ownership-is-growing-rapidly-around-the-world-but-not-always-equally/​ [accessed 2019-12-12]
    13. Kreuze E, Jenkins C, Gregoski M, York J, Mueller M, Lamis DA, et al. Technology-enhanced suicide prevention interventions: A systematic review. J Telemed Telecare 2017 Jul;23(6):605-617. [CrossRef] [Medline]
    14. Kreuze E, Ruggiero KJ. Technology-Oriented Suicide Prevention Interventions for Adolescents and Adolescent Gatekeepers: A Qualitative Review. Adolescent Res Rev 2017 Jun 6;3(2):219-233. [CrossRef]
    15. Christensen H, Batterham P, O'Dea B. E-health interventions for suicide prevention. Int J Environ Res Public Health 2014 Aug 12;11(8):8193-8212 [FREE Full text] [CrossRef] [Medline]
    16. Reyes-Portillo JA, Mufson L, Greenhill LL, Gould MS, Fisher PW, Tarlow N, et al. Web-based interventions for youth internalizing problems: a systematic review. J Am Acad Child Adolesc Psychiatry 2014 Dec;53(12):1254-1270.e5. [CrossRef] [Medline]
    17. Perry Y, Werner-Seidler A, Calear AL, Christensen H. Web-Based and Mobile Suicide Prevention Interventions for Young People: A Systematic Review. J Can Acad Child Adolesc Psychiatry 2016;25(2):73-79 [FREE Full text] [Medline]
    18. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    19. Young teens (ages 12-14 years of age). Center for Disease Control and Prevention(CDC). 2019.   URL: https://www.cdc.gov/ncbddd/childdevelopment/positiveparenting/adolescence.html [accessed 2019-12-12]
    20. World Health Organization (WHO). Adolescent health in the South-East Asia region. 2020.   URL: https://www.who.int/southeastasia/health-topics/adolescent-health [accessed 2019-12-12]
    21. Endnote. Citation and reference software. Philadelphia, PA: Clarivate Analytics; 2020.
    22. Covidence. Systematic review software. Melboure: Veritas Health Innovation; 2020.
    23. Gender in the Global Research Landscape. Elsevier. 2017.   URL: https://www.elsevier.com/research-intelligence/campaigns/gender-17 [accessed 2019-12-12]
    24. Ross E. Gender bias distorts peer review across fields. Nature News. 2017 Mar 23.   URL: https://www.nature.com/news/gender-bias-distorts-peer-review-across-fields-1.21685 [accessed 2020-05-01]
    25. Gabert R, Ng M, Sogarwal R, Bryant M, Deepu RV, McNellan CR, et al. Identifying gaps in the continuum of care for hypertension and diabetes in two Indian communities. BMC Health Serv Res 2017 Dec 27;17(1):846 [FREE Full text] [CrossRef] [Medline]
    26. Graboyes EM, Kompelli AR, Neskey DM, Brennan E, Nguyen S, Sterba KR, et al. Association of Treatment Delays With Survival for Patients With Head and Neck Cancer: A Systematic Review. JAMA Otolaryngol Head Neck Surg 2019 Feb 01;145(2):166-177 [FREE Full text] [CrossRef] [Medline]
    27. Tofighi B, Sindhu SS, Chemi C, Lewis CF, Dickson VV, Lee JD. Perspectives on the HIV continuum of care among adult opioid users in New York City: a qualitative study. Harm Reduct J 2019 Oct 12;16(1):58 [FREE Full text] [CrossRef] [Medline]
    28. Miller W, Brown J, Simpson T. What works? A methodological analysis of the alcohol treatment outcome literature. Needham Heights, MA: Allyn & Bacon; 1995:12-44.
    29. Miller W, Wilbourne PL. Mesa Grande: a methodological analysis of clinical trials of treatments for alcohol use disorders. Addiction 2002 Mar;97(3):265-277. [CrossRef] [Medline]
    30. Whiteman KL, Naslund JA, DiNapoli EA, Bruce ML, Bartels SJ. Systematic Review of Integrated General Medical and Psychiatric Self-Management Interventions for Adults With Serious Mental Illness. Psychiatr Serv 2016 Nov 01;67(11):1213-1225 [FREE Full text] [CrossRef] [Medline]
    31. Vaughn MG, Howard MO. Integrated psychosocial and opioid-antagonist treatment for alcohol dependence: A systematic review of controlled evaluations. Social Work Research 2004 Mar 01;28(1):41-53. [CrossRef]
    32. Cabassa LJ, Camacho D, Vélez-Grau CM, Stefancic A. Peer-based health interventions for people with serious mental illness: A systematic literature review. J Psychiatr Res 2017 Jan;84:80-89. [CrossRef] [Medline]
    33. Auslander W, Fisher C, Ollie M, Yu M. Teaching Master's and Doctoral Social Work Students to Systematically Evaluate Evidence-Based Interventions. Journal of Teaching in Social Work 2012 Sep;32(4):320-341. [CrossRef]
    34. Aseltine RH, DeMartino R. An outcome evaluation of the SOS Suicide Prevention Program. Am J Public Health 2004 Mar;94(3):446-451. [CrossRef] [Medline]
    35. Bailey E, Krysinska K, O'Dea B, Robinson J. Internet Forums for Suicide Bereavement. Crisis 2017 Nov;38(6):393-402. [CrossRef] [Medline]
    36. Freedenthal S. Adolescent help-seeking and the Yellow Ribbon Suicide Prevention Program: an evaluation. Suicide Life Threat Behav 2010 Dec;40(6):628-639. [CrossRef] [Medline]
    37. Haas A, Koestner B, Rosenberg J, Moore D, Garlow SJ, Sedway J, et al. An Interactive Web-Based Method of Outreach to College Students at Risk for Suicide. Journal of American College Health 2008 Jul;57(1):15-22. [CrossRef]
    38. Han J, Batterham PJ, Calear AL, Wu Y, Xue J, van Spijker BAJ. Development and pilot evaluation of an online psychoeducational program for suicide prevention among university students: A randomised controlled trial. Internet Interv 2018 Jun;12:111-120 [FREE Full text] [CrossRef] [Medline]
    39. Pisani AR, Wyman PA, Gurditta K, Schmeelk-Cone K, Anderson CL, Judd E. Mobile Phone Intervention to Reduce Youth Suicide in Rural Communities: Field Test. JMIR Ment Health 2018 May 31;5(2):e10425 [FREE Full text] [CrossRef] [Medline]
    40. Robinson J, Bailey E, Hetrick S, Paix S, O'Donnell M, Cox G, et al. Developing Social Media-Based Suicide Prevention Messages in Partnership With Young People: Exploratory Study. JMIR Ment Health 2017 Oct 04;4(4):e40 [FREE Full text] [CrossRef] [Medline]
    41. Schilling EA, Lawless M, Buchanan L, Aseltine RH. "Signs of Suicide" shows promise as a middle school suicide prevention program. Suicide Life Threat Behav 2014 Dec;44(6):653-667. [CrossRef] [Medline]
    42. Wyman PA, Brown CH, LoMurray M, Schmeelk-Cone K, Petrova M, Yu Q, et al. An outcome evaluation of the Sources of Strength suicide prevention program delivered by adolescent peer leaders in high schools. Am J Public Health 2010 Sep;100(9):1653-1661. [CrossRef] [Medline]
    43. Dickter B, Bunge EL, Brown LM, Leykin Y, Soares EE, Van Voorhees B, et al. Impact of an online depression prevention intervention on suicide risk factors for adolescents and young adults. Mhealth 2019;5:11 [FREE Full text] [CrossRef] [Medline]
    44. Hetrick SE, Goodall J, Yuen HP, Davey CG, Parker AG, Robinson J, et al. Comprehensive Online Self-Monitoring to Support Clinicians Manage Risk of Suicide in Youth Depression. Crisis 2017 May;38(3):147-157. [CrossRef] [Medline]
    45. Iorfino F, Davenport TA, Ospina-Pinillos L, Hermens DF, Cross S, Burns J, et al. Using New and Emerging Technologies to Identify and Respond to Suicidality Among Help-Seeking Young People: A Cross-Sectional Study. J Med Internet Res 2017 Jul 12;19(7):e247 [FREE Full text] [CrossRef] [Medline]
    46. King CA, Eisenberg D, Zheng K, Czyz E, Kramer A, Horwitz A, et al. Online suicide risk screening and intervention with college students: a pilot randomized controlled trial. J Consult Clin Psychol 2015 Jun;83(3):630-636 [FREE Full text] [CrossRef] [Medline]
    47. Bertolote JM, Fleischmann A, De Leo D, Phillips MR, Botega NJ, Vijayakumar L, et al. Repetition of suicide attempts: data from emergency care settings in five culturally different low- and middle-income countries participating in the WHO SUPRE-MISS Study. Crisis 2010;31(4):194-201. [CrossRef] [Medline]
    48. Czyz EK, King CA, Biermann BJ. Motivational Interviewing-Enhanced Safety Planning for Adolescents at High Suicide Risk: A Pilot Randomized Controlled Trial. J Clin Child Adolesc Psychol 2019;48(2):250-262. [CrossRef] [Medline]
    49. Hetrick SE, Yuen HP, Bailey E, Cox GR, Templer K, Rice SM, et al. Internet-based cognitive behavioural therapy for young people with suicide-related behaviour (Reframe-IT): a randomised controlled trial. Evid Based Ment Health 2017 Aug;20(3):76-82. [CrossRef] [Medline]
    50. King CA, Klaus N, Kramer A, Venkataraman S, Quinlan P, Gillespie B. The Youth-Nominated Support Team–Version II for suicidal adolescents: A randomized controlled intervention trial. Journal of Consulting and Clinical Psychology 2009 Oct;77(5):880-893 [FREE Full text] [CrossRef] [Medline]
    51. King R, Nurcombe B, Bickman L, Hides L, Reid W. Telephone counselling for adolescent suicide prevention: changes in suicidality and mental state from beginning to end of a counselling session. Suicide Life Threat Behav 2003;33(4):400-411. [CrossRef] [Medline]
    52. Mehlum L, Tørmoen AJ, Ramberg M, Haga E, Diep LM, Laberg S, et al. Dialectical behavior therapy for adolescents with repeated suicidal and self-harming behavior: a randomized trial. J Am Acad Child Adolesc Psychiatry 2014 Oct;53(10):1082-1091. [CrossRef] [Medline]
    53. Normand D, Colin S, Gaboulaud V, Baubet T, Taieb O. How to stay in touch with adolescents and young adults after a suicide attempt? Implementation of a 4-phones-calls procedure over 1 year after discharge from hospital, in a Parisian suburb. Encephale 2018 Sep;44(4):301-307. [CrossRef] [Medline]
    54. McManama O'Brien KH, LeCloux M, Ross A, Gironda C, Wharff EA. A Pilot Study of the Acceptability and Usability of a Smartphone Application Intervention for Suicidal Adolescents and Their Parents. Arch Suicide Res 2017;21(2):254-264. [CrossRef] [Medline]
    55. Asarnow JR, Baraff LJ, Berk M, Grob CS, Devich-Navarro M, Suddath R, et al. An emergency department intervention for linking pediatric suicidal patients to follow-up mental health treatment. Psychiatr Serv 2011 Nov;62(11):1303-1309 [FREE Full text] [CrossRef] [Medline]
    56. Tan Z, Liu X, Liu X, Cheng Q, Zhu T. Designing Microblog Direct Messages to Engage Social Media Users With Suicide Ideation: Interview and Survey Study on Weibo. J Med Internet Res 2017 Dec 12;19(12):e381 [FREE Full text] [CrossRef] [Medline]
    57. Yen S, Ranney ML, Tezanos KM, Chuong A, Kahler CW, Solomon JB, et al. Skills to Enhance Positivity in Suicidal Adolescents: Results From an Open Development Trial. Behav Modif 2019 Mar;43(2):202-221 [FREE Full text] [CrossRef] [Medline]
    58. Yen S, Spirito A, Weinstock L, Tezanos K, Kolobaric A, Miller I. Coping long term with active suicide in adolescents: Results from a pilot randomized controlled trial. Clin Child Psychol Psychiatry 2019;24(4):847-859. [CrossRef] [Medline]
    59. Silverstone PH, Bercov M, Suen VYM, Allen A, Cribben I, Goodrick J, et al. Long-term Results from the Empowering a Multimodal Pathway Toward Healthy Youth Program, a Multimodal School-Based Approach, Show Marked Reductions in Suicidality, Depression, and Anxiety in 6,227 Students in Grades 6-12 (Aged 11-18). Front Psychiatry 2017;8:81 [FREE Full text] [CrossRef] [Medline]
    60. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the Trial: Systematic Review of Real-World Uptake and Engagement With Digital Self-Help Interventions for Depression, Low Mood, or Anxiety. J Med Internet Res 2018 Jun 06;20(6):e199 [FREE Full text] [CrossRef] [Medline]
    61. Hermes ED, Lyon AR, Schueller SM, Glass JE. Measuring the Implementation of Behavioral Intervention Technologies: Recharacterization of Established Outcomes. J Med Internet Res 2019 Jan 25;21(1):e11752 [FREE Full text] [CrossRef] [Medline]
    62. Ash P. Children Are Different: Liability Issues in Working With Suicidal and Dangerous Youths. Focus (Am Psychiatr Publ) 2019 Oct;17(4):355-359. [CrossRef] [Medline]
    63. Thompson EC, Fox KA, Lapomardo A, Hunt JI, Wolff JC. Youth Self Report Thought Problems and Sleep Difficulties Are Linked to Suicidal Ideation Among Psychiatrically Hospitalized Adolescents. J Child Adolesc Psychopharmacol 2020 Feb 13. [CrossRef] [Medline]
    64. Van Meter AR, Algorta GP, Youngstrom EA, Lechtman Y, Youngstrom JK, Feeny NC, et al. Assessing for suicidal behavior in youth using the Achenbach System of Empirically Based Assessment. Eur Child Adolesc Psychiatry 2018 Feb;27(2):159-169 [FREE Full text] [CrossRef] [Medline]
    65. Van Roy B, Groholt B, Heyerdahl S, Clench-Aas J. Understanding discrepancies in parent-child reporting of emotional and behavioural problems: Effects of relational and socio-demographic factors. BMC Psychiatry 2010 Jul 16;10(1). [CrossRef]
    66. Berman NC, Sullivan A, Wilhelm S, Cohen IG. Effect of a legal prime on clinician's assessment of suicide risk. Death Stud 2016;40(1):61-67. [CrossRef] [Medline]
    67. Center for Disease Prevention and Control (CDC). 2018.   URL: https://www.cdc.gov/healthyyouth/data/yrbs/pdf/trendsreport.pdf [accessed 2020-12-12]
    68. Abrutyn S, Mueller AS. Toward a robust science of suicide: Epistemological, theoretical, and methodological considerations in advancing suicidology. Death Stud 2019 Sep 10:1-6. [CrossRef] [Medline]
    69. Maple M, Pearce T, Sanford RL, Cerel J. The Role of Social Work in Suicide Prevention, Intervention, and Postvention: A Scoping Review. Australian Social Work 2016 Aug 31;70(3):289-301. [CrossRef]
    70. Standley CJ. Expanding our paradigms: Intersectional and socioecological approaches to suicide prevention. Death Stud 2020 Feb 12:1-9. [CrossRef] [Medline]
    71. Witteman HO, Hendricks M, Straus S, Tannenbaum C. Are gender gaps due to evaluations of the applicant or the science? A natural experiment at a national funding agency. The Lancet 2019 Feb;393(10171):531-540. [CrossRef]
    72. Canetto S. Suicide as culturally scripted: Theory, evidence, and implications for prevention. In: Annual American Association Suicidology Conference. 2020 Presented at: American Association Suicidology; April 2020; Virtual conference.


    Abbreviations

    BIT: behavioral intervention technology
    MQRS: Methodological Quality Rating Scale
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    RCT: randomized controlled trial


    Edited by G Eysenbach; submitted 11.03.20; peer-reviewed by Y Perry, P Batterham, K McKay; comments to author 20.04.20; revised version received 26.05.20; accepted 25.06.20; published 09.10.20

    ©Hannah Szlyk, Jia Tan. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.10.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.