Published on in Vol 21, No 5 (2019): May

Preprints (earlier versions) of this paper are available at, first published .
Development and Evaluation of a Web-Based Resource for Suicidal Thoughts:

Development and Evaluation of a Web-Based Resource for Suicidal Thoughts:

Development and Evaluation of a Web-Based Resource for Suicidal Thoughts:

Original Paper, Seattle, WA, United States

2School of Social Work, University of Washington, Seattle, WA, United States

3Kaiser Permanente Washington Health Research Institute, Seattle, WA, United States

4Department of Health Services, University of Washington, Seattle, WA, United States

5Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States

6Berkeley School of Law, University of California, Berkeley, CA, United States

7Civilization, Seattle, WA, United States

8Prevention Communities, Fresno, CA, United States

Corresponding Author:

Ursula Whiteside, MS, PhD

2331 E Madison St

Seattle, WA, 98112

United States

Phone: 1 206 679 6349


Background: Nearly half of people who die by suicide see a health care provider in the month before their death. With the release of new care guidelines, detection of suicidal patients will likely increase. Providers need access to suicide-specific resources that can be used as part of immediate, brief interventions with a suicidal patient. Web-based suicide prevention resources have the potential to address this need.

Objective: This study aimed to describe the development of the website as a resource for individuals with suicidal thoughts and to evaluate the utility of the site via user experience surveys.

Methods: is an online video-based free public resource that provides evidence-based teachings, examples, and resources for managing suicidal thoughts and intense emotions focused largely around skills from dialectical behavior therapy. Developed with assistance from mental health consumers, it is intended to address gaps in access to services for suicidal patients in health care systems. Visitors stay an average of a minute and a half on the website. From March 2015 to December 2017, a user experience survey measured self-reported changes on a 1 (not at all) to 5 (completely overwhelming) scale regarding intensity of suicidal thoughts and negative emotions while on the website. Longitudinal regression analyses using generalized estimating equations evaluated the magnitude and statistical significance of user-reported changes in suicidal ideation and negative emotion. In secondary analyses, user-reported changes specific to subgroups, including men aged 36 to 64 years, mental health care providers, and other health care providers were evaluated.

Results: During the period of analysis, there were 138,386 unique website visitors. We analyzed surveys (N=3670) collected during that time. Subsamples included men aged 36 to 64 years (n=512), mental health providers (n=460), and other health care providers (n=308). A total of 28% (1028/3670) of survey completers rated their suicidal thoughts as a 5 or “completely overwhelming” when they entered the website. We observed significant reductions in self-reported intensity of suicidal thoughts (–0.21, P<.001) and negative emotions (–0.32, P<.001), including decreases for users with the most severe suicidal thoughts (–6.4%, P<.001), most severe negative emotions (–10.9%, P<.001), and for middle-aged men (–0.13, P<001). Results remained significant after controlling for length of visit to website (before the survey) and technology type (mobile, desktop, and tablet).

Conclusions: Survey respondents reported measurable reductions in intensity of suicidal thoughts and emotions, including those rating their suicidal thoughts as completely or almost completely overwhelming and among middle-aged men. Although results from this user-experience survey administered at one point in time to a convenience sample of users must be interpreted with caution, results provide preliminary support for the potential effectiveness of the website as a tool for short-term management of suicidal thoughts and negative emotions.

J Med Internet Res 2019;21(5):e13183




Nearly half of people who die by suicide in the United States see some type of health care provider in the month before their death [1,2]. Newly released screening and care guidelines for suicidal patients [3] have the potential to increase the number of suicidal patients detected across health care settings. Unfortunately, most providers, particularly those in primary care settings where the majority of patients are seen before death by suicide, generally are not trained to work with suicidal patients [4,5]. There is also a lack of widely disseminated resources available for immediate support of suicidal patients, and the vast majority of people who die by suicide never receive specialized mental health care [6], with men being particularly unlikely to seek mental health care but far more likely to die by suicide than women [7,8].

Providers need access to suicide-specific resources that can be used as part of immediate, brief interventions with a suicidal patient. Web-based suicide-prevention resources are available around the clock and have the potential to address several major challenges to caring for suicidal patients [9,10]. Web-based services can assist providers as they are providing care by giving them a free resource to share, support suicidal patients during the time between identification and receipt of specialty mental health care, and serve as an adjunct intervention for suicidal patients who are not able to access specialty metal health care or prefer not to receive this type of care (eg, males). However, in reviewing the existing literature on Web-based prevention strategies for suicidal individuals, Jacob et al explicitly noted the lack of evidence-based resources and called for further development and evaluation of these tools [11]. A recent review of the small number of studies on the effectiveness of Web-based intervention websites (and mobile apps) reported some promise in terms of reductions in suicidal thoughts [12], although many of the interventions reviewed included barriers to access such as the need to set up an account or were difficult or impossible to access outside of the specific research trial.

Web-based resources that do not require the user to sign-in (ie, provide contact information or other details about their identity) can reduce important obstacles to use, including concerns about privacy. In the United States, examples of highly used, free Web-based resources include sites from the American Foundation for Suicide Prevention [13] and the National Suicide Prevention Lifeline [14]. Both of these sites provide important resources and self-help tips for suicide prevention but are not specifically tailored for a patient and clinician audience. The effectiveness of educational Web resources for suicide and self-directed violence prevention has historically been an area of research need [15]. Recent research indicates positive findings, for example, Till et al [16] reported results from a randomized trial that indicated that users with higher baseline scores of suicidality experienced an immediate reduction of suicidal thoughts compared with persons who visited a control website, thus indicating promise for the use of educative websites for brief intervention. Moreover, free Web-based resources may be particularly useful for middle-aged men in the United States who are unlikely to seek mental health care [17] and disproportionately impacted by suicide death [18].

Objectives is a website that was developed based on the evidence provided above, for the purpose of supporting suicidal patients and providers who care for them ([19], see Multimedia Appendix 1). Central to the educative website are first-person stories of successful coping with and recovery from suicidal thoughts and painful emotions using strategies from a highly effective treatment for suicidal individuals (dialectical behavior therapy, DBT) [20-22]. Indeed, research indicates that video portrayals of recovery from suicide-related crisis may be beneficial to those experiencing higher rates of suicidal thoughts [23]. The purpose of this report was to describe the development and evaluation of this Web-based resource. For our evaluation, we present data from a brief user-experience survey collected from users. The survey measures changes in suicidal thoughts and negative emotions while on the website and also collects demographic data on a high-risk group (men aged 36-64 years) and health care provider status. We hypothesized that survey completers, including those who rated their suicidal thoughts and negative emotions at the most intense end of the scale, would report reduced intensity after viewing the website. Secondary stratified analyses evaluated the impact on a smaller and higher risk group (middle age men) and providers (mental health and other health care providers) who may be using the website to help support suicidal patients rather than self-help. In addition, we investigated length of time to take the survey as an indicator of website “dose” as the length of exposure to the website may be associated with ratings of intensity of suicidal thoughts and negative emotions, and the impact of viewing the website on a mobile phone versus other device to inform our recommendations for website use. Origins

DBT is a robust evidence-based treatment that includes skills and self-management strategies (DBT skills) to change and tolerate emotions that drive suicidal thoughts [20-22,24]. However, frontline clinical providers such as those working in primary care have few resources to explain or connect patients to specialized mental health treatment, including DBT. Mental health consumer advocates have created websites to make DBT resources more widely available, for example, [25]. When brief DBT-based interventions began to demonstrate promise for treatment of suicidal patients [26-28], the first and second authors (motivated as researchers but also by personal and professional experiences–related suicidal patients, their providers, and perceived lack of resources) initiated research to begin the development of a DBT-inspired Web-based resource [19,29]. The results were used to design the website. This was done in collaboration with Team Now Matters Now, a group of mental health consumers with suicidal experiences recruited for the project and the website design firm Civilization [30]. Most of the website content consists of videos of Team Now Matters Now members discussing their lived experience including using DBT skills to manage suicidal thoughts, thus providing social models of success [31]. The website was launched on World Suicide Prevention Day in 2014 (September 10) and had more than 250,000 unique visitors as of December 2018. Design

The website is designed to provide people experiencing suicidal thoughts and health care providers with DBT skills and to give health care providers a resource for supporting suicidal patients. The long-term goal is having providers use the skills in their own life and teach them to patients to help prevent and manage suicidal crises. As such, the website was developed with the provider audience in mind as a tool to help provide better care to their suicidal patients. The landing page states “Have you had suicidal thoughts? Problems that felt unsolvable? You are in excellent company—we’ve been there.” This statement was intended to reduce stigma and signify that others having similar experiences were part of the website. Users proceed via a button to the home page. This page presents a panel of images of individuals with lived experience who have videos on the website. Images lead to topics such as DBT skills (opposite action, mindfulness, mindfulness of current emotion, paced breathing), suicidal thoughts (videos linking suicidal thoughts to difficulties managing emotions and DBT skills), and lethal means and caring messages (topics central to current care guidelines; Figure 1) [32]. Clicking on text in individual panels leads to videos on the topic with personal stories and didactic content. Home page menu options are About (text-based description of the website and a brief video provide an overview of the website), Team (bios and photos of Team Now Matters Now members allowing visitors more context about those featured in the videos), and More (eg, additional engagement options such as social media links, email sign-up, and downloadable “caring message” cards that providers can share with patients to direct them to the site). The home page also has a Crisis Lines option leading to national and international phone, short message service text message, and instant messaging options (also part of care guidelines). is built on WordPress and instrumented with Google Analytics, which collects client-side statistics for how users find and interact with the site.

User Experience Survey regularly collects survey data during a user’s session for quality improvement purposes and to determine to what degree certain populations use the website and if it helps them. The 7-item survey was built in WordPress using HTML elements triggered by JavaScript, is not advertised, and open (no log-in required). The header is, “We’d appreciate your confidential feedback to improve our site.” The survey asks users to retrospectively rate their “Intensity of negative emotions” and “Intensity of suicidal thoughts” as 1 to 5 (“Not at all” to “Completely Overwhelming;” Figure 2) for “When you entered this site” (baseline) and “What level are they now?” (post website use). Only 1 number can be selected for each question. These survey items were adapted from DBT skills “diary cards” used by patients to track their suicidal thoughts, mood, and related behaviors [20-22]. The survey also asks about demographics as nonexclusive checkboxes (ie, male aged 36-64 years, mental health provider, and health care provider). Survey responses were not assessed for completeness nor required for survey submission (although only complete surveys were used in analyses), and once submitted (using a “submit” button), all responses were final. Data that were entered, but not submitted, were not recorded. The methods of survey development were similar to best practices recommended by the US Department of Health and Human Services for developing Web-based surveys such as simplicity, clarity, and user centeredness [33]. The survey asks about suicidal thoughts and negative emotions, which likely correspond [34].

The survey displays after 1 min on any page viewed on mobile phones (timing determined by the survey software). On desktops and tablets, it originally displayed after 3 min (the approximate length of 1 short video) on a page and changed to 8 min (essentially the length of 2 short videos or 1 longer video) on August 7, 2015 to allow for longer time on the website before surveying users. We know that participants visited a page for the minimum time to receive the survey, but they could have visited several pages before triggering the survey. Without using cookies, we could track only how long users were on a page, not the website, and we are reluctant to use tracking features because of the sensitive website content. Once the survey surfaces, users cannot proceed until they fill out the survey, close its window using a button in the right-hand corner, or choose another option on the navigation bar. Once displayed (whether completed or not), the survey does not surface for that user for at least 24 hours. We do not know the number of times the survey surfaced and was not completed. The survey was tested by Civilization website developers before release and before data collection for this study started. We had no contact with the responders other than their survey submission.

Figure 1. Website.
View this figure
Figure 2. Survey.
View this figure

Study Sample and Study Data

This study analyzed a convenience sample of data from surveys completed from March 5, 2015, the first full day the survey was posted, to December 3, 2017, when data were retrieved. The personal information of Internet Protocol (IP) address snippets was removed from the dataset after assigning each participant a unique identification number. The final dataset was stored in Google Drive, to which authors UW, RN, AJW, XZ, DH had access. We did not track how long survey completers stayed on the website beyond the survey. The Allendale Institutional Review Board (Old Lyme, Connecticut) approved this study.

Statistical Analyses

Primary Outcome

Longitudinal regression analyses used generalized estimating equations (GEEs; [35]) to evaluate the magnitude and statistical significance of user-reported changes in (1) suicidal ideation and (2) negative emotion. An exchangeable working correlation matrix in conjunction with cluster robust standard errors was estimated to account for correlation of repeated measures (baseline and post) within participants. A total of 2 sets of analyses were performed evaluating outcomes for both suicidal thoughts and negative emotions as (1) a continuous measure and (2) a dichotomous indicator of high (4 or 5) versus low ratings (3 or less). Logistic and Gaussian GEEs were used for binary and continuous variables, respectively. In the primary analyses evaluating changes in outcomes across all participants, each outcome variable was regressed on time point (post vs baseline).

Men Aged 36 to 64 Years and Health Care Providers

To evaluate if changes in outcomes differed by these demographic groups, primary analyses were extended to include indicator variables for (1) middle-aged male (=1), (2) mental health provider (=1), (3) other health provider (=1), and (4) the interaction of each demographic variable with time (post vs baseline). In stratified analyses, each outcome variable was regressed on middle-aged male, mental health provider, other health provider, middle-aged male by time (post vs baseline), mental health provider by time, and other health provider by time. The statistical test of the subgroup effects was the magnitude and statistical significance of each demographic variable by time interaction.

Mobile Versus Other Device and Delay to Survey

To evaluate if outcomes differed by electronic device type or delay before being presented with the survey, primary analyses were extended to include indicator variables for (1) device type (nonmobile vs mobile) and (2) survey delay interval (1, 3, 8 min). For device-type analysis, outcome variables were regressed on device type, time (post vs baseline), and the device type by time interaction. In delay-interval analysis, outcome variables were regressed on delay interval, time, and the delay interval by time interaction. Delay interval was divided into 2 planned contrasts of (1) 8 versus 1 min and (2) 3 versus 1 min. Statistical tests of device-type effects and delay-related effects were magnitude and statistical significance of (1) device type by time and (2) delay interval by time interactions.

Website Visitors

During the time in which surveys were analyzed, 167,878 sessions (website visits) and 138,386 unique visitors were recorded. Of those sessions, 52% came to the website via “organic search terms,” 21% came directly to the website (bookmark or typing in the URL), 9% came via a link from social media, 9% came via paid advertisements for the site (Google AdWords), and 9% came via referrals (link from another website). On average, users viewed 2.1 pages per session and stayed on the website for 1 min 31 seconds. The average bounce rate (leaving after viewing 1 page) was 61%, and 18% of sessions were from returning visitors. People arrived via organic search terms when they searched for something that led to being listed, and then clicked on our link, and the link was not a paid advertisement. To better understand how people found the website, Google search term data collection was enabled from December 11, 2017 to February 3, 2018 (54 days), which provided results that indicated that a number of organic search terms related to suicide and suicide help (eg, “crisis chat,” “crisis text line uk,” “suicide chat,” and “now matters now,” each individually making up no more that 4% of searches) represented the search terms leading to The Google Adwords were phrases we thought people who were suicidal might Google (“I want to die” and “how do I kill myself?”). If someone Googled such a term, they might see the website as an advertisement option. Geographically, visitors were largely from the United States (61%), Canada (15%), United Kingdom (9%), and Russia (3%), with Australia, Brazil, New Zealand, India, France, and Germany each making up 2% or less.

Survey Responders

We received 3931 complete responses. Of these, 261 were removed because they were likely from people who previously completed the survey, identified from duplicate IP address snippets (keeping only earliest dated record). Surveys with incomplete data were not included in the analyses as the amount of missing data was minimal (6.7%) [36]. Of the 3670 unique survey responses, 514 (14% of survey sample) identified as men aged 36 to 64 years. Of survey completers, 460 (13%) identified as mental health professionals and 308 (8%) as other health care professionals, with 40 (1%) identifying as both. Users completed surveys after 1-min (2096; 57%), 3-min (388; 11%), or 8-min (1186; 32%) delays. Mobile users (57%) all received the survey at 1 min, therefore nonmobile device users comprised the remaining 43% (1574) of the sample (Table 1).

Survey ratings for suicidal thoughts pooled at the low and high ends of the scale, clustering toward low (1, not at all) and high (4 and 5, completely overwhelming). For baseline, the greatest number of responses were a score of 5 (28%), followed by 1 (28%), 4 (21%), 3 (17%), and 2 (7%). Post scores were 1 (32%), 2 (29%), 5 (23%), 4 (19%), and 3 (17%). In our sample, of those who reported suicidal thoughts (2644) at baseline, 29% (n=763) reported a 1 or more point reduction in suicidal thoughts pre versus post visit, the vast majority stayed the same (63%, n=1666), and 8% (n=215) rated suicidal thoughts 1 or more points worse (8%). Of those who reported worse scores, 2% (n=51) were more than a point worse, 1% (n=21) were 3 or 4 points worse.

Primary Outcome Analyses

The overall changes reported for all participants for the 4 main study outcomes (continuous and dichotomous for baseline vs post) is shown in Figure 3. Survey participants reported an average 0.21-point reduction (95% CI –0.24 to –0.19; P<.001; d=0.13) in suicidal ideation and a 0.32-point reduction (95% CI –0.35 to –0.29; P<.001; d=0.21) in negative emotion ratings (both 1 to 5 scale) after viewing the website. The average visitor’s probability of endorsing high suicidal ideation (4 or 5) decreased 6.4 percentage points (95% CI –7.55 to –5.33; P<.001; d=0.13) after viewing the website. The average visitor’s probability of endorsing high negative emotion (4 or 5 at baseline) decreased by 10.9 percentage points (95% CI –12.20 to –9.64; P<.001; d=0.23) after viewing the website.

Table 1. Demographic groups, electronic device type, and delay to survey (N=3670).
Outcome variableTotalMen aged 36 to 64 years (n=512)Mental health provider (n=460)Other health care provider (n=308)Mobile (all 1 min to survey) (n=2096)8 min to survey (n=1186)3 min to survey (n=388)
Baseline suicidal thoughts

Intensity, mean (SD)3.1 (1.6)a3.3 (1.6)a1.8 (1.5)2.8 (1.7)3.6 (1.4)a2.7 (1.6)a2.2 (1.5)a

Percentage low (score=1), n (%)1012 (28)131 (26)331 (72)130 (42)315 (15)486 (41)215 (55)

Percentage high (score=4-5), n (%)1784 (49)a277 (54)a86 (19)122 (40)1235 (59)a448 (38)a101 (26)a
Baseline negative thoughts

Intensity, mean (SD)3.7 (1.5)a3.7 (1.5)a2.2 (1.6)a3.1 (1.7)a4.1 (1.3)a3.3 (1.6)a2.8 (1.6)a

aIndicates significant reduction from baseline to post P<.001, not applicable from baseline suicide ideation.

Figure 3. Change in negative emotion and suicidal ideation (continuous and dichotomous) from baseline to post website for all participants. Error bars indicate SDs.
View this figure
Men Aged 36 to 64 Years

Men aged 36 to 64 years reported their overall severity of suicidal ideation decreased on the 1 to 5 scale (Estimate, est=–0.13; 95% CI –0.19 to –0.07; P<.001; d=–0.08), although the decrease was 0.09 points greater (d=0.06; P<.001) among nonmiddle-aged males (est=-0.22; 95% CI –0.25 to -0.20; P<.001; d=0.14). High suicidal ideation decreased for middle-aged males (%=–4.74; 95% CI –7.38 to –2.10; P<.001; d=0.09). Results were similar and significant for overall negative emotions (est=–0.21; 95% CI –0.29 to –0.14; P<.001; d=0.14) and for the probability of high negative emotion (%=–6.90; 95% CI –10.70 to –3.11; P<.001; d=0.14; Figure 4).

Figure 4. Average suicidal ideation and high suicidal ideation ratings from baseline to post website by demographic group. MH: mental health.
View this figure
Health Care Providers

For mental health providers, there were no changes observed for overall severity of suicidal ideation from baseline to post (est=–0.03; 95% CI –0.07 to 0.01; P=.10; d=0.02) or for the probability of endorsing high suicidality (%=–1.36; 95% CI –2.86 to 0.14; P=.08; d=0.03). This finding was consistent with other health care providers for suicide ideation overall (est=–0.07; 95% CI –0.15 to 0.02; P=.12; d=0.04) and high suicidal ideation (%=–0.54; 95% CI –3.95 to 2.87; P=.76; d=0.01). However, significant improvements were seen in negative emotions overall for mental health providers (est=–0.18; 95% CI –0.24 to –0.12; P<.001; d=0.12) and high negative emotions (%=–4.17; 95% CI –6.39 to –1.95; P<.001; d=0.09) as well as other health care providers (Overall: est=–0.18; 95% CI –0.29 to –0.07; P=.001; d=0.12 and high rating: %=–5.59; 95% CI –10.04 to –1.14; P=.01; d=0.12; Figure 4).

Mobile Versus Other Device and Delay to Survey

For website platform (mobile vs other), suicidal ideation severity decreased for both desktop or tablet users (est=–0.21; 95% CI –0.25 to –0.17; P<.001; d=0.13) and mobile users (est=–0.21; 95% CI –0.25 to –0.18; P<.001; d=0.13). High suicidal ideation decreased for both desktop or tablet users (%=–6.99; 95% CI –8.59 to –5.38; P<.001; d=0.14) and mobile users (%=–6.06; 95% CI –7.59 to –4.52; P<.001; d=0.12). Similarly, negative emotion severity decreased for both desktop or tablet users (est=–0.34; 95% CI –0.38 to –0.29; P<.001; d=0.23) and mobile users (est=–0.30; 95% CI –0.34 to –0.26; P<.001; d=0.20). In addition, high negative emotions decreased for both desktop or tablet users (%=–10.67; 95% CI –12.55 to –8.79; P<.001; d=0.22) and mobile users (percentage=–11.26; 95% CI –13.07 to –9.45; P<.001; d=0.24).

For survey delay interval (1, 3, 8 min), suicidal ideation severity decreased for those surveyed after delays of 1 min (est=–0.21; 95% CI –0.25 to –0.18; P<.001; d=0.13), 3 min (est=–0.18; 95% CI –0.24 to –0.12; P<.001; d=0.11), and 8 min (est=–0.22; 95% CI –0.26 to –0.18; P<.001; d=0.14). High suicidal ideation also decreased for those surveyed after delays of 1 min (%=–6.06; 95% CI –7.59 to –4.52; P<.001; d=0.12), 3 min (%=–5.67; 95% CI –8.38 to –2.96; P<.001; d=0.11), and 8 min (%=–7.42; 95% CI –9.36 to –5.48; P<.001; d=0.15). Similarly, negative emotion severity decreased for surveyed after delays of 1 min (est=–0.30; 95% CI –0.34 to –0.26; P<.001; d=0.20), 3 min (est=–0.29; 95% CI –0.37 to –0.21; P<.001; d=0.19), and 8 min (est=–0.35; 95% CI –0.40 to –0.30; P<.001; d=0.24). High negative emotions also decreased for those surveyed after delays of 1 min (%=–11.26; 95% CI –13.07 to –9.45; P<.001; d=0.24), 3 min (%=–10.05; 95% CI –13.37 to –6.74; P<.001; d=0.21), and 8 min (%=–10.88; 95% CI –13.12 to –8.63; P<.001; d=0.23).

Sensitivity Analyses Correcting for Multiple Statistical Tests

All findings (P<.05) remained statistically significant after accounting for multiple testing using the Benjamini-Hochberg procedure [37].

Principal Findings

The results of our evaluation of suggest that this Web-based resource has the potential to address several major challenges to caring for suicidal patients. First, was developed to address a historic need for suicidal patients [15] by providing tailored content to support suicidal individuals and the providers who care for them [19]. During our 32-month evaluation period, there were 138,386 unique visitors to, indicating this site was indeed receiving a fair amount of visitation. Next, over 70% of survey completers recalled having some degree of suicidal thoughts when they arrived at the website. Survey participants, overall, also reported significant reductions in suicidal thoughts. Highly suicidal respondents reported significant decreases in intensity of suicidal thoughts from website entrance to survey. Reductions in suicidal thoughts were matched by similar declines in reported intensity of negative emotions while visiting the website. This result indicated that the site attracted people who are experiencing suicidal thoughts, often at a high level of intensity. The results also suggest that website visits were associated with reductions in suicidal thoughts and negative emotions.

Finally, we examined different types of visitors and found that significant reductions in suicidal thoughts and negative emotions were largely consistent across subgroups. Middle-aged men reported significant reductions in suicidal thoughts and negative emotions. Given that this group makes up 38% of all suicides [35], these results are promising. However, reductions for middle-aged men were not as large as for the rest of the sample; therefore, the website could use greater tailoring toward men. For example, male identification on the survey could trigger an additional box requesting suggestions for improvements or their contact information for follow-up interview or survey about the same topic. About one-fifth of responders identified as mental health or other health care providers, which was overrepresented compared with the general population. These responders reported lower levels of suicidal thoughts and negative emotions. Health care providers likely have visited the website for reasons other than self-help, such as for providing a referral for a patient, as the website is part of clinician training efforts. These findings suggest that the website content is most useful for those for whom it is directly intended: those experiencing suicide ideation. Given the intentional simplicity of the survey, we could not glean information about the motivations for initial visits to the website (eg, experiencing severe negative emotions vs visiting for work). In the future, the survey could be developed to have an opt-out option for health care providers or those visiting the website for reasons not related to their own mental health, but this may also create unintentional stigma toward having suicidal thoughts. When we looked separately at user platform (mobile, desktop, and tablet), positive reductions were found in each group. We found this result to be promising because it may mean the website was showing benefit for youth, given that mobile users tend to be younger [38]. Indeed, the demographic category “age 12-18” was added to the user-experience survey in 2018, and results of these are forthcoming. Interestingly, mobile device users had higher severity ratings across both time points. Therefore, we may seek ways to target mobile audiences who, based on self-report, may be at greater risk for suicide. When we looked at time on website pages before the survey surfaced (1, 3, or 8 min), positive reductions were also consistent across times. This result meant that some users may find benefit in as little as 1 min on the website. We cannot separate the 1-min survey and mobile findings, but both were positive for reducing suicidal ideation.


There are important limitations to this study. The primary limitation is that user-experience data were collected at one point in time from individual users, without a control-group comparison. Distraction alone may have had an effect, or the passage of time may be associated with similar reductions. In addition, we do not know if experienced improvements lasted beyond the brief period that the survey covered. In addition, nonresponse bias may account for the findings—these findings may not generalize to people who saw the survey but did not complete it or people who never saw the survey who may not have had positive responses. It is possible that the nature of the survey topic (suicide) may have made some individuals less likely (eg, among those with greater self-stigma or experienced discrimination) or more likely (eg, experiencing the survey as highly relevant) to participate, including those potentially at higher risk of suicide. Another limitation is that participants were asked to rate their pre and post website suicidal thoughts and negative emotions on a single survey. Although this format was designed to maximize survey response, reported reductions in distress may be at least partially attributable to recall bias. Furthermore, the survey has face validity, but other types of validity have not been examined, and we do not know if survey completers represent typical suicidal patients seen in health care settings, our target audience. Finally, although changes in suicidal thoughts and emotions might occur through a number of mechanisms (eg, didactic information, exposure to evidence-based interventions, distraction from emotional suffering, referral to crisis services, return to baseline, placebo effect, and viewing videos of those with lived experiencing discussing stories of recovery), we could not evaluate these mediators given survey limits. Future randomized trials need to examine if this website (or which elements of this website), compared with other sites, is associated with similar or greater reductions in suicidal thought reductions. An in-progress randomized controlled trial comparing risk of suicide attempt (over 18 months) among patients reporting suicidal ideation will shed additional light on the utility of its content. Patients invited to a skills training intervention, utilizing a guided interactive version the website, will be compared with patients randomized to usual care [39].

Conclusions is a Web-based resource with the potential to help address gaps in suicide prevention and treatment resources for mental health professionals. Our findings supported use of the website as originally intended—as a support tool for providers who identify suicidal patients in their practice. In light of our results indicating website use is associated with reductions in suicidal thoughts, we suggest that health care providers can, with some confidence, refer suicidal people for short-term management of suicidal thoughts and negative emotions. For example, when suicidal patients are discovered as part of routine screening for depression, providers could share this website with the patient as part of safety planning procedures, which includes a list of options for managing suicidal thoughts [40]. Indeed, a number of health care providers are directed to the website through the Zero Suicide initiative, which has trained over 500 health systems to provide better suicide care. The resource is shared in nearly all of these trainings [41].


The authors would like to gratefully acknowledge the great number of volunteer hours that resulted in this work. Funding for the development of the website came from an R-34 grant from the National Institutes of Mental Health (MH097836) and a Postdoctoral Fellowship (011811) from the American Foundation for Suicide Prevention awarded to Dr UW. Funding for Ms JR’s time was provided by a development fund award from Kaiser Permanente Washington Health Research Institute. Funding for Ms RH and Dr DH was provided by small private donations to

Conflicts of Interest is a free public resource. It also is a 501c3 nonprofit in the United States. Authors UW, RN, DL, AJW, XZ, and DQL volunteer for this organization and none have ever been paid. RH and DH are contract employees who worked on this paper. In 2014, ME’s company (Civilization) helped develop the website and branding. There is no conflict of interest, but there are co-interests of better access to support for people at risk for suicide.

Multimedia Appendix 1 introduction video.

MP4 File (MP4 Video), 31MB

  1. Ahmedani BK, Simon GE, Stewart C, Beck A, Waitzfelder BE, Rossom R, et al. Health care contacts in the year before suicide death. J Gen Intern Med 2014 Jun;29(6):870-877 [FREE Full text] [CrossRef] [Medline]
  2. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry 2002 Jun;159(6):909-916 [FREE Full text] [CrossRef] [Medline]
  3. National Action Aliance for Suicide Prevention. 2018. Transforming Health Systems Initiative Work Group   URL: https:/​/theactionalliance.​org/​sites/​default/​files/​action_alliance_recommended_standard_care_final.​pdf [accessed 2019-04-21] [WebCite Cache]
  4. Jahn D, Quinnett P, Ries R. The influence of training and experience on mental health practitioners’ comfort working with suicidal individuals. Prof Psychol Res Pr 2016;47(2):130-138. [CrossRef]
  5. Schmitz WM, Allen MH, Feldman BN, Gutin NJ, Jahn DR, Kleespies PM, et al. Preventing suicide through improved training in suicide risk assessment and care: an American Association of Suicidology Task Force report addressing serious gaps in U.S. mental health training. Suicide Life Threat Behav 2012 Jun;42(3):292-304. [CrossRef] [Medline]
  6. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv 2013 Dec 1;64(12):1195-1202 [FREE Full text] [CrossRef] [Medline]
  7. Parent M, Hammer J, Bradstreet T, Schwartz E, Jobe T. Men's mental health help-seeking behaviors: an intersectional analysis. Am J Mens Health 2018 Jan;12(1):64-73 [FREE Full text] [CrossRef] [Medline]
  8. Centers for Disease Control and Prevention, National Center for Health Statistics. Suicide and Self-Inflicted Injury. 2019.   URL: [accessed 2019-03-13] [WebCite Cache]
  9. Till B, Tran US, Voracek M, Niederkrotenthaler T. Beneficial and harmful effects of educative suicide prevention websites: randomised controlled trial exploring Papageno v Werther effects. Br J Psychiatry 2017 Aug;211(2):109-115. [CrossRef] [Medline]
  10. van Spijker BA, Werner-Seidler A, Batterham PJ, Mackinnon A, Calear AL, Gosling JA, et al. Effectiveness of a web-based self-help program for suicidal thinking in an Australian community sample: randomized controlled trial. J Med Internet Res 2018 Feb 14;20(2):e15 [FREE Full text] [CrossRef] [Medline]
  11. Jacob N, Scourfield J, Evans R. Suicide prevention via the internet: a descriptive review. Crisis 2014 Jan 1;35(4):261-267. [CrossRef] [Medline]
  12. Witt K, Spittal MJ, Carter G, Pirkis J, Hetrick S, Currier D, et al. Effectiveness of online and mobile telephone applications ('apps') for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysis. BMC Psychiatry 2017 Dec 15;17(1):297 [FREE Full text] [CrossRef] [Medline]
  13. American Foundation for Suicide Prevention.   URL: [accessed 2011-05-10] [WebCite Cache]
  14. National Suicide Prevention Lifeline.   URL: [accessed 2018-12-19] [WebCite Cache]
  15. Luxton DD, June JD, Fairall JM. Social media and suicide: a public health perspective. Am J Public Health 2012 May;102(Suppl 2):S195-S200. [CrossRef] [Medline]
  16. Till B, Tran US, Voracek M, Niederkrotenthaler T. Beneficial and harmful effects of educative suicide prevention websites: randomised controlled trial exploring Papageno v. Werther effects. Br J Psychiatry 2017 Aug;211(2):109-115. [CrossRef] [Medline]
  17. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. Leading Causes of Death Reports, 1981-2017. 2016.   URL: [accessed 2019-04-21] [WebCite Cache]
  18. House J, Marasli P, Lister M, Brown JS. Male views on help-seeking for depression: a Q methodology study. Psychol Psychother 2018 Dec;91(1):117-140. [CrossRef] [Medline]
  19. Whiteside U, Richards J, Ellsworth M. NowMattersNow. 2014.   URL: [accessed 2018-12-18] [WebCite Cache]
  20. Linehan MM. DBT Skills Training Handouts And Worksheets. New York: The Guilford Press; 2014.
  21. Linehan MM. Cognitive-behavioral Treatment Of Borderline Personality Disorder. New York: The Guilford Press; 1993.
  22. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry 2006 Jul;63(7):757-766. [CrossRef] [Medline]
  23. Till B, Strauss M, Sonneck G, Niederkrotenthaler T. Determining the effects of films with suicidal content: a laboratory experiment. Br J Psychiatry 2015 Jul;207(1):72-78. [CrossRef] [Medline]
  24. Linehan MM, Korslund KE, Harned MS, Gallop RJ, Lungu A, Neacsiu AD, et al. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis. JAMA Psychiatry 2015 May;72(5):475-482. [CrossRef] [Medline]
  25. Dietz L. DBT Self Help. 2003.   URL: [accessed 2018-12-18] [WebCite Cache]
  26. Ward Ciesielski EF. An open pilot feasibility study of a brief dialectical behavior therapy skills-based intervention for suicidal individuals. Suicide Life Threat Behav 2013 Jun;43(3):324-335. [CrossRef] [Medline]
  27. Whiteside U. A Brief Personalized Feedback Intervention Integrating a Motivational Interviewing Therapeutic Style and Dialectical Behavioral Therapy Skills for Depressed Or Anxious Heavy Drinking Young Adults. Seattle: University of Washington; 2009.
  28. McMain SF, Guimond T, Barnhart R, Habinski L, Streiner DL. A randomized trial of brief dialectical behaviour therapy skills training in suicidal patients suffering from borderline disorder. Acta Psychiatr Scand 2017 Feb;135(2):138-148. [CrossRef] [Medline]
  29. Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, et al. Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res 2014 Feb 7;16(2):e42 [FREE Full text] [CrossRef] [Medline]
  30. Ellsworth M. Civilization Design Firm.   URL: [accessed 2018-12-18] [WebCite Cache]
  31. Bandura A. Principles of Behavior Modification: Social Learning Theory. New York: Holt, Rinehart and Winston, Inc; 1969.
  32. National Action Alliance for Suicide Prevention. Washington, DC: Education Development Center Recommended standard care for people with suicide risk: making health care suicide safe   URL: https:/​/theactionalliance.​org/​sites/​default/​files/​action_alliance_recommended_standard_care_final.​pdf [accessed 2019-04-21] [WebCite Cache]
  33. US Department of Health & Human Services. Online Surveys   URL: [accessed 2019-02-06] [WebCite Cache]
  34. Hinkin T. A brief tutorial on the development of measures for use in survey questionnaires. Organ Res Methods 2016 Jun 29;1(1):104-121. [CrossRef]
  35. Liang K, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika 1986;73(1):13-22. [CrossRef]
  36. Schafer J. Multiple imputation: a primer. Stat Methods Med Res 1999 Mar;8(1):3-15. [CrossRef] [Medline]
  37. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B 2018 Dec 5;57(1):289-300. [CrossRef]
  38. Anderson M, Jiang J. Public Services Alliance. 2018. Teens, social media & technology   URL: http:/​/publicservicesalliance.​org/​wp-content/​uploads/​2018/​06/​Teens-Social-Media-Technology-2018-PEW.​pdf [accessed 2019-04-19] [WebCite Cache]
  39. Simon GE, Beck A, Rossom R, Richards J, Kirlin B, King D, et al. Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial. Trials 2016 Dec 15;17(1):452 [FREE Full text] [CrossRef] [Medline]
  40. Stanley B, Brown G. Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract 2012;19(2):256-264. [CrossRef]
  41. Goldstein-Grumet J. Suicide Prevention Resource Center. Zero Suicide   URL: [accessed 2019-04-08] [WebCite Cache]

DBT: dialectical behavior therapy
GEE: generalized estimating equation
IP: Internet Protocol

Edited by G Eysenbach; submitted 19.12.18; peer-reviewed by SH Kwon, E Bradshaw, C Wilks, E Ward-Ciesielski; comments to author 17.01.19; revised version received 14.03.19; accepted 12.04.19; published 02.05.19


©Ursula Whiteside, Julie Richards, David Huh, Rianna Hidalgo, Rebecca Nordhauser, Albert J Wong, Xiaoshan Zhang, David D Luxton, Michael Ellsworth, DeQuincy Lezine. Originally published in the Journal of Medical Internet Research (, 02.05.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.