Participants with a prior confirmed suicide attempt were recruited from an ongoing randomized clinical trial [41]. Inclusion criteria were (1) inpatient or emergency service admission, (2) lifetime suicide attempt and current hospital admission for suicidality or current hospital admission for a suicide attempt, and (3) consenting to study procedures. This study was approved by the institutional review board at the University of Washington. Data collection occurred from November 2017 to October 2019, and data analysis was conducted from October 2019 to August 2020.

Participants were asked to complete a 30-minute semistructured interview about their concerns and suggestions for using internet search data as a means of preventing suicide. They were offered the option to provide a one-time confidential data download of their online Google search history. Participants who opted to participate in the study were reimbursed US $30 regardless of whether or not they agreed to share their Google search history.

Google Takeout is a web-based interface developed by Google that allows users of Google apps to download their data into an exportable file. Using prior work [42], we created a web app, called gTAP [43], to allow participants to download their data without sharing personal Google account credentials. Only past Google search history was collected (Multimedia Appendix 1).

The Suicide Attempt and Self-Injury Count, a brief version of the Suicide Attempt Self-Injury Interview [44], has been widely used to determine suicide attempts in clinical trials [45-49] and was used to identify dates of suicide events and categorize events into suicide attempts and nonsuicidal acts.

The interview was developed by the research team and focused on the acceptability of using internet information to detect the risk of suicide and to prevent suicide (Multimedia Appendix 2). Participants were asked to respond to the following question: “Technology companies use algorithms to predict who is at risk for suicide. Were you aware of this? Do you have any concerns or fears about how this information is collected, stored, and shared? How would you feel if they used your personal search data and/or what you have posted on social media to take action to prevent you and others in a similar situation from suicide? What do you see as the pros and cons?”

Participants’ web searches were used to generate behavioral (online information seeking pattern) and semantic (meaning of search content) features. For behavioral features, we generated daily summary of participants' search history such as the average number of searches per day and the time of day when searches were conducted. For semantic features, we applied distributional models of semantics [50] to derive vector representations of participants’ queries, such that queries and words relating to similar concepts would be proximal in the resulting vector space. To do so, we used semantic vectors [51-53] and publicly available pretrained word embeddings [54] (Multimedia Appendix 3 and Multimedia Appendix 4). In order to map the user search queries to 9 empirically supported warning signs [12] and suicide-method preparation (which has not been previously examined using the present approach), we developed a set of cue terms. An iterative process of cue term definition, expert review of proximal queries, and refinement of the cue term set was used to generate the final set. We calculated a proximity score between the participants' search query (web search) and vector subspace derived from the cue terms representing a warning sign, using the Gram-Schmidt orthonormalization [55] procedure to ensure mutual orthogonality. The proximity score was estimated as the length of the projection of the query into this subspace, following the quantum disjunction method [56]. We used a conservative z score threshold ≥3.5 (corresponding P=.0002) to indicate a meaningful construct-to-query association relative to all search queries for each participant. The use of a threshold of this nature is required on account of the statistical properties of high-dimensional space—all vectors in this space have measurable similarity, but only proximal neighbors in the space indicate meaningful similarity [55,57,58], which should be well above what would be anticipated by chance on account of the high probability of randomly instantiated vectors being mutually close-to-orthogonal [59,60]. Furthermore, this threshold was supported empirically by inspection of the relationships between queries that fell above and the themes of interest in a subset of the data (Multimedia Appendix 4).

Because of a small sample size and high level of heterogeneity in web search data across participants, we used an individualized analytical approach (Multimedia Appendix 4) to assess the magnitude of difference between web search data proximal (7, 15, 30, and 60 days) to the suicide attempts versus the typical (baseline) search behavior of participants. In order to show the difference between the observed search feature score prior to the time of a suicide attempt and a typical baseline of the search feature for an individual, we computed 2 versions of a standardized value for the search score at the time of the event. First, we presented the z score defined as the search score at the observed event time (based on a chosen time window), standardized using the mean and variance associated with a patient-specific reference distribution characterizing the full range of observed search scores for an individual. This highlighted how far the observed score at the event time was from typical scores. Second, given that the reference search score distributions may be highly skewed or multimodal, we also computed the empirical reference percentile for each search feature for the event time (suicide attempt). Specifically, we used an empirical reference distribution per search feature to compute the percent of typical individual-specific search feature values that fell either above or below the corresponding search feature value for an observed attempt. A percentile calculation is a form of nonparametric standardization and is termed a placement value for classification problems [61]. To show either extremely low or high values, we calculated the symmetrized placement value defined as the minimum of the upper or lower tail probability for the observed attempt score. The participant-specific reference distributions were generated using a nonparametric Monte Carlo simulation [62]. All statistical analysis was performed using the R [63] statistical programming language.

Participants’ responses to structured survey questions were summarized using summary statistics. Semistructured responses were transcribed verbatim during participant interviews and anonymized prior to analysis. We used a mixed methods approach, combining quantitative and qualitative data with the function of expansion [64], which allowed inductive qualitative data to provide the why to questions uncovered by the quantitative data. Qualitative data were analyzed from a constructivist perspective using thematic analysis after all interviews were completed [65]. Two coders (a psychologist and a psychiatrist) independently extracted initial themes from survey responses. Themes were reviewed by a third coder, a clinical research assistant who had performed all participant interviews. Data were iteratively reviewed (open coded) and collapsed to mutually exclusive themes (axial coding) until saturation was achieved (ie, when no new themes emerged) [66]. Triangulation [67] of quantitative and qualitative data allowed for convergence of themes and a more comprehensive understanding of willingness to use search data for prevention. Illustrative quotes and themes are provided for a qualitative data audit trail. This study was conducted in accordance with SRQR (Standards for Reporting Qualitative Research) [68].

In total, search history data for 24,397 days were collected with 349,922 individual search queries from 26 study participants. The median time span of the search history data across participants was 1348 days (range 220-4752 days); however, the actual number of days when participants searched online was much lower than the data collection period and varied widely (median 898.5 days, range 75-2759 days) (Figure 2a). Analysis generated 11 high-level search constructs related to participants’ online information-seeking behavior (ie, search behavior features) and search content proximity scores for 10 suicide warning signs (ie, semantic features) (Multimedia Appendix 3).

Our analysis revealed idiosyncratic online information seeking behavior across individuals; the number of searches per day varied between 2 searches and 24 searches (median 8.5 searches). The time of day when participants conducted online searches also varied (morning: 0%-4.35%; late night: 0%-37.5%; Figure 2b,c).

Of the semantic search content that we mapped to known suicide warning signs, we identified a small proportion of search queries (median 1.2%, range 0.06%-21.47%) with a proximity z score ≥3.5 that mapped onto 1 or more of the 10 warning signs. At times, queries meeting this threshold were observed in close proximity to a suicide attempt (within days to weeks). Table 2 provides a representative selection of highly ranked queries (based on z scores) for each warning sign, revealing content suggestive of premeditation, preparation, substance use, stressors, underlying mental state, and help-seeking behavior.