Individuals seen for mental health treatment at Group Health, a large health care organization based in Seattle, Washington, are asked to complete the Patient Health Questionnaire (PHQ-9) before every visit as part of routine care. The PHQ-9 is a 9-item, self-report questionnaire used for screening, monitoring, and measuring the severity of depression over the previous 2 weeks [24]. The PHQ-9 uses a 4-point scale (“not at all”, “several days”, “more than half the days”, “nearly every day”), and the last question asks about presence and duration of suicide ideation. An answer of “more than half the days” or “nearly every day” for suicide ideation has been found to predict subsequent suicide attempts or suicide death [25].

In March 2013, clinic intake staff distributed flyers announcing the study, “Your Feedback Counts”, to potential participants (individuals waiting to be seen for mental health appointments) together with the PHQ-9. The flyer explained that a score of two or three (“more than half the days” or “nearly every day” respectively) on the suicide ideation question made them eligible to provide feedback on components of a suicide prevention intervention under development. The patient could access an anonymous online survey by following a link. After providing consent online, participants completed the anonymous survey.

As a last step, participants could link to a separate survey and provide their contact information to receive a US $10 incentive. No link between survey data and contact information was collected. All study procedures were approved by the Group Health Institutional Review Board.

The survey included questions about demographics and medical treatment for a suicide attempt or self-injury, as well as acceptability of proposed intervention messaging (see Multimedia Appendices 1 and 2). A patient consultant group of 5 individuals with a history of suicidal ideation and suicide attempts helped to generate the messaging options, as well as a video describing the intervention. There were 26 options for message subject lines introducing the intervention, 20 options for intervention characteristics, 9 intervention brand names, and an option for study participants to make additional suggestions. The survey was programmed using DatStat Ilume and accessible by computer or other portable Internet-connected device (smartphone, iPad, etc) [26]. All questions related to the intervention invitation and description were asked using a 5-point Likert scale (0=strongly disagree; 4=strongly agree). In evaluating the results, we calculated the difference in responders endorsing “agree” or “strongly agree” for a particular option and “disagree” or “strongly disagree”. Descriptive statistical analyses were conducted via SPSS 16 [27].

Of the 39 participants who visited the online consent page, 38 agreed to participate. Three individuals consented but provided no further information and four individuals provided only demographic information. Participants completed the survey in an average of 20.5 minutes (SD 25.98). Most participants were female, had received medical help for a suicide attempt or self-injury during their lifetime, and were under 64 years old (Table 1).

Of the 26 different subject lines proposed, 5 emerged as agreeable to more than 45% of responders and disagreeable to less than 25%—“Checking In”, “Touching Base”, “Between Visits Resources”, “Can We Help?”, and “Invitation to Online Support Program”. Messages advertising support as “free” or “quick” (including words as “instant”, “free”, “simple”) were less appealing to participants (Figure 1).

Twenty intervention characteristics were provided, and an average of 26.5 participants answered each question (Figure 2). The most preferred characteristics were that the program was personalized, that someone who cares personally is involved, that others have found this program helpful, that real people have found the program helpful, and that “you are not alone”. Participants least preferred a “slick” looking and “well made” program.

The two most preferred options for the intervention name brand were “Now Matters Now” and “You Are Here”. Preference for intervention name varied widely with an agree/disagree gap between –37.5% (“Tiny Matters”) and 30.4% (“Now Matters Now”). Each question was answered on average by 23.5 responders. See Figure 3.

Survey participants were asked to watch a 2-minute video about the intervention and answer a series of 8 follow-up questions (see Figure 4). The purpose of the video evaluation was to identify specific aspects of the video that might need improvement. Participants were asked about technical characteristics (eg, quality of sound) and artistic impression (eg, music, visuals) as well as content itself (eg, liking what the speaker had to say).

The main goal of this project was to survey patients with current suicidal ideation about acceptable engagement messaging for an online intervention for individuals with suicidal thoughts. Invitation subject lines, intervention descriptions, project names, and an introductory video were reviewed. The most agreeable subject lines for invitation messages were simple inquiries about how the respondent was doing (“Checking in” and “Touching base”). Also agreeable were messages highlighting additional “resources” that “your provider thought would be helpful”. Certain messaging appeared less agreeable to participants, such as messages that proposed something “free” or “instant”. Intervention characteristics that respondents endorsed were personalization, caring, and real people with personal examples. Less important characteristics were appearance and something “new and exciting”. Two brand names relating to the present moment, “Now Matters Now” and “You Are Here”, were rated most agreeable. Overall, most respondents agreed that they could relate to the video introduction, but responses also indicated room for improvement with technical aspects and video comprehension. This project accomplished the goal of giving researchers a good place to start with messaging designed to promote uptake and engagement in an online intervention. Perhaps, more importantly, we now have a better idea about messaging options to avoid.

There are limitations to this project. The sample size was small and demographically narrow and thus did not allow for subgroup analyses or comparisons. However, the study sample demographics were similar to those who most commonly attempt suicide in our health care setting [25]. Individuals in the United States most likely to die by suicide, that is, males in their middle and late years, are not represented here. Future research should apply such branding and messaging questions to this specific population, but also a much larger sample in general, given the large deviations in preferences for intervention name/brand. We chose not to advertise more broadly for this survey (on websites or with advertising, which may have resulted in a larger sample) because we were interested specifically in the population our intervention is targeted to—those at increased risk for suicide attempt in the following year in our health care setting [25].

To preserve the anonymity of responders we asked only basic demographic questions. We do not know how many patients were potentially eligible to participate, so we cannot determine the response rate. Participants were informed of inclusion criteria and then were self-selected into the study, which means there may be potential selection bias. We recruited a treatment-seeking population from a mental-health clinic waiting room, and the target population of the future intervention also includes patients receiving only primary care treatment who may have different messaging preferences. We focused on the brevity of the survey, which limited our ability to explore systematic testing or manipulation of messaging content. An alternative explanation for the findings may be that the messages containing “instant/free/simple” were too vague, rather than the terms themselves being unappealing. Last, the survey options were not based on a theoretical foundation or prior hypotheses due to a lack of prior research targeting suicidal individuals for online interventions.

Future research should include systematic manipulation of variations around messaging (eg, comparing “Between visits resources” with “Between visits free resources”) in the hope of providing greater understanding of the effectiveness of the various message components. Future research including larger samples and such manipulations could allow us to determine how interventions might be delivered differently for subgroups of the target population (depending on symptoms, preferences, etc).

We consulted with a group of patients with self-reported suicidal thoughts and past attempts to help generate the options we presented to patients. Patients with mental illnesses and suicidal ideation or behavior often report discrimination on many levels (personal, community, institutional, etc) [28]. Receiving feedback from our patient consults was particularly important since we wanted to avoid messaging that would reinforce the stigma of mental health illnesses and prevent or discourage people affected by mental illness from seeking treatment. This paper provides one model (including development of an engagement survey, audience for an engagement survey, methods for presenting results of an engagement survey) for including target users in the development of uptake strategies for online mental health interventions. Finally, large pragmatic clinical trials should be conducted in order to identify whether an online intervention could reduce suicide attempts or deaths on a large scale.