This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Treatment for suicidality can be delivered online, but evidence for its effectiveness is needed.
The goal of our study was to examine the effectiveness of an online self-help intervention for suicidal thinking compared to an attention-matched control program.
A 2-arm randomized controlled trial was conducted with assessment at postintervention, 6, and, 12 months. Through media and community advertizing, 418 suicidal adults were recruited to an online portal and were delivered the intervention program (Living with Deadly Thoughts) or a control program (Living Well). The primary outcome was severity of suicidal thinking, assessed using the Columbia Suicide Severity Rating Scale.
Intention-to-treat analyses showed significant reductions in the severity of suicidal thinking at postintervention, 6, and 12 months. However, no overall group differences were found.
Living with Deadly Thoughts was of no greater effectiveness than the control group. Further investigation into the conditions under which this program may be beneficial is now needed. Limitations of this trial include it being underpowered given the effect size ultimately observed, a high attrition rate, and the inability of determining suicide deaths or of verifying self-reported suicide attempts.
Australian New Zealand Clinical Trials Registry ACTRN12613000410752; https://www.anzctr.org.au/ Trial/Registration/TrialReview.aspx?id=364016 (Archived by WebCite at http://www.webcitation.org/6vK5FvQXy); Universal Trial Number U1111-1141-6595
Suicidal thinking is common and often precedes suicidal plans and attempts [
A Dutch trial of a self-guided online intervention for suicidal thinking reported significant reductions in suicidal thoughts relative to the waitlist control condition that included access to a website with psychoeducational material about suicide [
The study was a 2-arm randomized controlled trial delivered entirely online consisting of a treatment condition, Living with Deadly Thoughts (LwDT), and an attention-matched control condition, Living Well. There were 4 measurement occasions: baseline, postintervention (6 weeks after baseline) and 6- and 12-month follow-up. Full details on the study methodology can be found in the trial protocol [
Participants were recruited between November 2013 and December 2015 through online media forums including websites, social networking websites, and advertizing on search engines. A link to a welcome screen included an invitation to provide consent and complete the online screening procedure.
Eligibility criteria were as follows: aged 18 to 65 years, valid email address, access to a reliable internet connection, located in Australia, fluent in English, no history of a diagnosed psychotic disorder, currently experiencing suicidal thoughts, and no suicide attempts in the past month. These were assessed using single, self-report questions. Unlike the original trial [
Respondents who did not meet inclusion criteria were redirected to a “thank you” page listing referral information. Respondents who were excluded based on a recent suicide attempt were also provided with the opportunity to submit their phone number to receive a phone call from the Suicide Call Back Service (SCBS), a 24/7, Australia-wide, not-for-profit service that provides telephone counseling (www.suicidecallbackservice.org.au). Participants were informed that the study was not intended to replace treatment as usual and were encouraged to seek or continue other treatment.
LwDT is an adapted but closely aligned translation of the Dutch Web-based program
The program consists of 6 online modules. Participants are instructed to complete 1 module per week and to spend 30 minutes per day using the program. Each module contains 4 components: (1) theory, (2) a weekly assignment, (3) 2 to 3 exercises, and (4) optional exercises to help consolidate relevant information and skills. The modules become available in a fixed sequence, 4 days after the start of the previous module, regardless of completion, and remain available throughout the intervention period. Access to referral information via a “get help now” link is available on every Web page. A safety procedure included monitoring and intervention (if required) by the SCBS (see Safety Procedures).
The attention control condition was matched to the active program in length, style, and availability. It involved a 6-week modular online learning course containing lifestyle information on (1) nutrition, (2) a healthy home environment, (3) a healthy weight, (4) a healthy heart, (5) healthy skin, and (6) a healthy mouth. Participants in the control condition received the safety protocol procedure, including monitoring and intervention (if required) by the SCBS (see Safety Procedures).
The primary outcome was severity of suicidal thinking, assessed using the Intensity of Suicidal Ideation subscale of the Columbia Suicide Severity Rating Scale (C-SSRS) [
Secondary outcome measures included presence of ideation and behavior, measured by the Suicidal Ideation and Suicidal Behavior subscales of the C-SSRS, respectively; reasons for living, using the Brief Reasons for Living Scale [
Standard demographic information including sex, age, relationship status, education, and employment status were collected together with self-reported lifetime suicide attempts and adherence to program by website usage.
Given the vulnerable, at-risk population that this study targeted, safety procedures were designed in collaboration with the SCBS with the goal of protecting and assisting participants while enabling them to maintain anonymity within the trial and considering ethics and clinical obligations. The ability to remain autonomous in the decision to make contact with services as much as possible was also important, as this is consistent with the concept of self-help and patient empowerment.
The safety procedures required eligible participants to make contact with the SCBS during enrollment to obtain a unique identification code, so that codes rather than names could be used in all subsequent communication between the research team and SCBS. On each measurement occasion, the first 3 items of the intensity of suicidal ideation section of the C-SSRS were used to detect high risk. Scores above a specified cutoff (a score of 5 on any of these items) alerted the participant to contact the SCBS. Not doing this within 2 days triggered a reminder email to the participant, as well as a message to the SCBS asking them to contact the participant (if contact information was available, otherwise no further action was taken). The provision of contact information to the research team or to SCBS was a voluntary option at enrollment for the trial.
Conversations between participants and SCBS staff (counselors, social workers, and psychologists) were conducted according to SCBS protocols without input from the research team.
Power to detect change in suicidal thoughts on the C-SSRS was based on an effect size of 0.3 (Cohen
Randomization to the active or control condition occurred on a 1:1 ratio using a block design (4 participants per block), stratified by sex and severity of suicidal thinking (high severity was defined as endorsing yes on the fifth item of the Suicidal Ideation subscale of the C-SSRS, which assesses active suicidal ideation, defined as having a specific plan and intent). The randomization procedure was incorporated into the website and was fully automated.
No research personnel were involved in the delivery of the interventions. Participants were not informed whether their assigned condition would receive the active or control program. However, it is likely that participants were able to discern their allocation based on the nature of the intervention they received, which increases risk of detection bias. All outcome measures were self-report and completed via an online portal. All research personnel (except those involved with the day-to-day management of the trial), remained blind to intervention allocation.
Ethical approval was obtained from the human research ethics committees of the University of New South Wales (HC13117) and the Australian National University (2012/471). The trial was registered at Australia New Zealand Clinical Trials (ACTRN12613000410752) and has a Universal Trial Number (U1111-1141-6595).
Following the screening process, eligible participants received instructions specifying that participation required making contact with the SCBS to obtain a unique identification code. Respondents provided consent, a valid email address, a name/nickname, and a telephone number (nonmandatory). Participants were then invited to complete baseline measures after which they were randomized.
Six weeks after randomization (after completing the program), participants were invited to complete the postintervention questionnaire. Follow-up assessments took place at 6- and 12-month follow-up. Participants who stopped using the program but who did not formally withdraw continued to be emailed follow-up questionnaires. Participants in the control group were given access to LwDT after the 12-month follow-up assessment.
Primary analyses were performed on an intention-to-treat basis. All outcomes were tested using planned contrasts of mean change scores from pre- to postintervention via a mixed-model repeated measures (MMRM) analytic approach. An unstructured variance-covariance matrix was used to model within-individual dependencies. MMRM analysis uses all available data and yields estimates of effect under restricted maximum likelihood. The test of the efficacy of the intervention was based on the interaction between intervention condition and time. Supplementary analyses on the primary outcome were conducted for individuals who completed more than half of the program, as defined by completion of 4 or more of the available 6 modules. Using the whole sample, moderation of the effectiveness of the intervention by sex, age, baseline depression severity, history of suicide attempts (nil vs at least one), and chronicity of suicidal thinking (total number of months spent thinking about suicide) was examined. Moderation was tested using an identical MMRM approach with the addition of the moderating variable of interest as a factor into the model with 2- and 3-way interactions with group and time [
Participant flow is shown in
Sample characteristics are provided in
On average, the severity of suicidal thinking fell in the very high range, with one-third of the sample reporting multiple past suicide attempts. Symptoms of psychopathology were elevated, including depression, hopelessness, anxiety, panic, and sleep disturbance. Compared to normative samples, health-related quality of life was lower and impairment was higher. There were no group differences on baseline clinical variables, with the exception of the intensity of suicidal thinking, which was higher in the intervention group relative to the control group,
Attrition was 45.9% (192/418) at postintervention, 65.3% (273/418) at 6 months, and 66.3% (277/418) at 12-month follow-up (see
In terms of adherence to the program, there were no significant between-group differences in the number of modules completed, χ26=9.26,
Participants who scored above the cutoff on the C-SSRS at any time during the study were telephoned by the suicide crisis help-line. More participants in the intervention condition (59/207) were called compared to those in the control condition (37/211), χ21=7.10,
Based on self-report data from the C-SSRS Suicidal Behavior subscale, 23 participants made a suicide attempt during the course of the study, with no differences between groups, χ21=.03,
There was no overall group difference between the LwDT group and the control group on the primary outcome, severity of suicidal thinking at postintervention,
Participant flow diagram.
Sample characteristics of participants in the intervention (Living with Deadly Thoughts) and control (Living Well) groups.
Characteristic | Intervention (n=207) | Control (n=211) | ||
Gender (female), n (%) | 160 (77.3) | 163 (77.3) | .37 | |
Age, years, mean (SD) | 39.53 (11.94) | 41.73 (11.86) | .06 | |
Educational history (completed secondary school), n (%) | 158 (76.3) | 158 (74.9) | .42 | |
Marital status (married or de facto), n (%) | 87 (42.0) | 73 (34.6) | .27 | |
Employment (employed), n (%) | 121 (58.5) | 127 (60.2) | .47 | |
Metropolitan | 127 (61.4) | 126 (59.7) | .95 | |
Regional, rural, or remote | 79 (38.2) | 84 (39.8) | .63 | |
Never | 90 (43.5) | 101 (47.9) | .37 | |
Once | 46 (22.2) | 43 (20.4) | .65 | |
More than once | 71 (34.3) | 67 (31.8) | .58 | |
Accessed at least 4 modules | 71 (34.3) | 88 (41.7) | .14 |
Alerts per group.
Number of alerts | Intervention (n=207) | Control (n=211) |
1 | 34 | 25 |
2 | 15 | 8 |
3 | 7 | 1 |
4 | 2 | 2 |
5 | 1 | 1 |
However, both groups showed significant reductions in the severity of suicidal thinking relative to baseline at postintervention,
Participants who completed at least 4 of the 6 modules in the LwDT condition were compared to those who completed at least 4 modules of the control program. Analyses indicated that completers of the LwDT program experienced a reduction in severity of suicidal thinking relative to completers of the control program at posttest,
Moderation of the effectiveness of the intervention by sex, age, baseline depressive symptoms, history of attempted suicide, and chronicity of suicidal thinking was investigated by examining 3-way interactions between group, time, condition, and the moderating variable of interest. Severity of suicidal thinking was not moderated by sex,
The LwDT program, compared with the LW program, had no significant effect on secondary outcomes except for anxiety (see
Across both groups (LwDT and Living Well) there was a significant reduction at all time points in the following: suicidal ideation (as measured by the C-SSRS and by the SIDAS), suicidal behavior, burdensomeness, depression, hopelessness, anxiety, panic, physical health, mental health, and physical functioning. There were also significant decreases in both groups for thwarted belongingness at 6- and 12-month follow-up, rumination at 6- and 12-month follow-up, and sleep difficulties at posttest and 12-month follow-up. Test statistics for these within-group comparisons are available from the authors upon request.
Variables that did not significantly improve over time were acquired capability for suicide, reasons for living, and alcohol use, although a group difference emerged at 12-month follow-up, with those in the intervention group having lower alcohol use scores than those in the control group,
Regardless of intervention allocation, participants’ level of suicidal thinking reduced over time, with no difference between the groups at postintervention or 6- or 12-month follow-up. Also, there was no evidence of between-group differences on most of the secondary variables.
These findings are discrepant from the original study of LwDT, where a significant difference was found for the intervention compared to a waitlist control group in suicidal thoughts and worry [
A second difference is that the original study involved a waitlist control condition, while the present trial had an attention-control condition. A number of features of the control intervention, including the fortnightly monitoring during the active phase, risk-triggered alerts, proactive follow-ups (if required) by the SCBS, and delivery of helpful lifestyle materials may have contributed to the reduction in suicidality, in addition to an expected standard placebo effect. The proactive nature of the intervention in the control group may have contributed to the improvement over time, obscuring any addition impact of the LwDT intervention. The monitoring of suicide ideation itself combined with the intervention by the call back service provides a safety protocol procedure that is known to have an effect [
A key procedural aim of this study was to provide LwDT without requiring participants to register with their doctors. Lack of anonymity poses an obstacle to help-seeking for many individuals [
A related issue was attrition and the role that general practitioners may have played in the original trial. In this study, attrition was 46% at postintervention, which is at the higher end of rates of attrition of online interventions [
The difference in number of alerts in this trial between the groups was also notable, with significantly more alerts triggered in the intervention group. This difference was not observed in the Dutch trial. Reasons for this difference are speculative but may be due to an imbalance of high-need people between the groups (as intensity of suicidal ideation was higher at baseline in the intervention group). Alternatively, the high alert rate may indicate that the intervention has an effect on help-seeking from a telephone helpline—something that is actively encouraged in the LwDT program. As registration with a helpline was mandatory for this trial, this may have lowered the threshold for calling, a factor that did not apply in the Dutch trial.
The current trial had a number of other limitations. First, the target sample size for the trial was not met due to time constraints. The study was therefore underpowered and, given the high attrition rate, any smaller differences would not have been detected. Determination of any deaths by suicide was not possible within the context of the trial, given the lack of access to official records and anonymity of the participants. While this trial was not powered to detect group-level differences in attempted suicides or suicide deaths, future trials in this field recruiting larger numbers of participants would do well to incorporate an analysis of patient records to assess deaths by suicide over an extended period. Further, the incidence of suicide attempts could not be verified given the reliance on self-report rather than hospital and medical records.
The results of this study indicate that completing an online intervention is associated with reduced suicidal thinking and psychological symptoms over time, but this may be due to the exposure to a structured program, monitoring, and safety procedures. Further assessment with a waitlist control group would be required to confirm any advantage of the structured program above that of the safety procedures, and such a trial is unlikely to receive ethical approval.
This study has several clinical and research implications. Overall, there was no evidence that the LwDT program is harmful and, looking to the original Dutch trial, may be beneficial to those who have a less chronic history of suicidal thinking. This is consistent with clinical guidelines for the use of low-intensity interventions (ie, National Institute for Health and Care Excellence guidelines) for individuals who have low to moderate symptom levels. However, with the Dutch trial indicating more pronounced effects for people with a history of suicide attempt and this trial showing less effectiveness for those with a longer history of suicidal thinking, future research should investigate what target group might benefit most from online self-help, what adjustments might need to be made to the program to accommodate for various degrees of suicidality, and how the delivery of the program might improve outcomes at both ends of the spectrum. For example, incorporation in a stepped care model or tailoring of the program to individual needs might increase effectiveness across levels of severity.
Furthermore, the results that were achieved are of interest because they suggest that interventions are associated with a drop in symptoms. Research in support of crisis lines is also built on this type of outcome study and used to support their effectiveness. Also notable, with respect to helplines, is that even though it would make sense to implement programs such as LwDT in organizations that provide crisis support, less than half of eligible respondents enrolled with the SCBS, indicating that this could also be a deterrent. This should be taken into account when preparing such programs for use outside of a research context. Finally, the type of high-quality trial methodology—a randomized controlled trial—such as ours may not be the best methodology to observe outcomes, given the constraints outlined above.
We have shown that the provision of a self -help program via the internet is feasible but that further research is needed to better understand the types of interventions, both online and offline, that could assist people experiencing suicide ideation and the most useful settings in which to situate these interventions.
CONSORT‐EHEALTH checklist (V 1.6.1).
Estimated marginal means from mixed-model repeated measures for primary and secondary outcomes at each time point for the intervention condition.
Columbia Suicide Severity Rating Scale
Living with Deadly Thoughts
mixed-model repeated measures
National Health and Medical Research Council
Suicide Call Back Service
Suicidal Ideation Attributes Scale
Sincere thanks to Anna Frayne and Kale Dyer, who provided support in the management of the trial. This study was supported by a grant from the Australian National Health and Medical Research Council (NHMRC; GNT1046317) and forms part of the program of research conducted by the NHMRC Centre of Research Excellence in Suicide Prevention (GNT1042580). The funder does not have any role or ultimate authority in the study design; collection, management, analysis, and interpretation of data; writing of the report; or the decision to submit the report for publication. AC is supported by NHMRC Early Career Fellowship 1122544. PB is supported by NHMRC Early Career Fellowship 1083311. HC is supported by NHMRC Fellowship 1056964.
HC and BAJvS designed the study and oversaw trial implementation, data collection, and interpretation. BAJvS and AJFMK conducted the original study in the Netherlands on which the current trial was based. BAJvS, HC , PJP, AM, AJFMK, and JAG obtained study funding. AWS and DS managed the trial. AWS conducted statistical analyses that were overseen by the trial statisticians, PJP and AM. FS provided clinical support on the trial. AWS and HC wrote the manuscript. All authors have read and provided revisions of the final manuscript.
BAJvS and AJFMK are authors of the Dutch Web-based program described in this manuscript. BAJvS and AJFMK are also authors of and receive royalties from an adapted paper version of the self-help program published under the title Piekeren Over Zelfdoding [