Over 700,000 people die by suicide each year globally [1]. Despite considerable research examining suicide risk factors, suicidal behavior remains difficult to predict. While the link between suicide and mental health difficulties (such as depression and alcohol use disorders) is well established in high-income countries, many suicides occur in moments of crisis with a breakdown in the ability to cope with difficulties, such as relationship breakups or chronic pain and illnesses. Experiencing conflict, disaster, violence, abuse or loss, and a sense of isolation are strongly associated with suicidal behavior [1]. Suicide rates are also high among vulnerable groups who experience discrimination, such as refugees and migrants; Indigenous peoples; lesbian, gay, bisexual, transgender, and intersex persons; and prisoners [1]. A meta-analysis of 50 years of research called for a shift in focus from risk factors (eg, previous suicide attempt and substance use) to machine learning (ML)–based risk algorithms [2]. Theoretical advances in the suicide literature propose dynamic relationships between key factors that warrant innovative research methodologies for adequate investigations [3-6]. In parallel, real-time assessment of suicidal thoughts and behaviors and related variables, using ecological momentary assessment (EMA) [7,8], offers a means of providing moment-to-moment assessment, more suitable for investigating the complex interplay between psychological, social, biological, and cultural factors put forward by prominent theories [3,9,10].

EMA uses systematic and frequent assessments to gather momentary data regarding an individual’s thoughts, emotions, behaviors, physical symptoms, and contexts as they occur in real time and outside clinical and other controlled settings. Common categories of EMA include (1) diaries, (2) experience sampling, (3) event-based sampling [11], and (4) passive sensing through wearable devices [7]. Diaries typically assess experience in fixed intervals (often daily) and involve retrospective recall; experience sampling typically uses a device to signal to the respondent to report on an experience at random times throughout the day. Event-based sampling elicits reports at the time of a particular event. Passive EMA involves measuring physiological states such as sleep or physical activity, typically (but not necessarily) via electronic wearable devices [7].

Such assessments are often delivered using handheld devices (such as smartphones) capable of time-stamping responses to short self-report questions in daily life. These measurements provide many methodological advantages over conventional assessment strategies when studying variable phenomena, investigating intraindividual changes, or complex and dynamic relationships between variables [7]. In addition, recall biases encountered in conventional retrospective survey methods can be diminished by the momentary evaluation of experiences or behaviors. Ultimately, such timely assessments could inform immediate and individually tailored interventions, offering advantages such as accessibility, availability, and versatility [12].

EMA designs have been found to be feasible and acceptable when delivered online and to anonymous participants [13] and have yielded good predictive validity in the measurement of suicide-specific outcomes when compared to standardized measures [14]. Indeed, more individuals reported suicidal ideation through EMA than traditional (retrospective) self-report measures. A recent systematic review reported that EMA of suicidal thoughts and behaviors was acceptable for participants and did not increase risk [14]. Available models suggest that through the process of active reflection and self-monitoring, social desirability, or feedback processes, the intensity of symptoms may decrease due to the unusual attention on target symptoms. Frequent self-monitoring may be a generic active ingredient across a variety of evidence-based psychological interventions (eg, diary cards in dialectical behavior therapy). However, researchers have noted that EMA components are inconsistently reported [14]. On the basis of the review by Kivelä et al [14], it is advisable to prioritize more frequent but brief assessments over short time periods to establish higher compliance. Future research should aim to more systematically examine how increasing the number of daily prompts affects compliance rates in order to establish optimal sampling schedules that balance temporal coverage with participant burden. Gee et al [15] asserted that further research should use longitudinal study designs, harmonize datasets, and use ML techniques to identify patterns of proximal risk factors for suicide behaviors.

Using data from smartphones and wearable devices, incorporating active data (active input from the user) or passive data (input from sensors), to better understand human behavior and personalize treatment plans is often described as digital phenotyping. This moment-by-moment quantification of individual-level human phenotype in situ, using data from personal digital devices [16], is being explored in relation to addiction, borderline personality disorder (BPD), posttraumatic stress disorder, and, more recently, suicidality [15-19].

Reviews of artificial intelligence (AI) in suicide research completed by Bernert et al [20] and later by Lejeune et al [21] highlight the burgeoning research in this area and demonstrate the potential that AI holds for identifying individuals at risk of suicide. AI can be used to identify patterns in large datasets, generate risk algorithms, and determine the effects of risk and protective factors on suicide.

As a form of AI, ML strategies enable computer learning of advanced classifiers to improve the accuracy of suicide prediction using large datasets. Burke et al [22] identified three main goals of ML studies of suicide: (1) improve the accuracy of risk prediction, (2) identify important predictors and the interactions between them, and (3) model subgroups of patients. Studies focused on improving suicide risk prediction suggest the high predictive potential for this technology [23]. Researchers have used supervised learning models [24], including ensemble learning methods (eg, random forests [RFs]), naïve Bayes classification, logistic and least squares regression, decision trees (DTs), and support vector machines, as well as unsupervised learning models [25], such as neural networks, clustering algorithms, self-organizing maps, principal component analysis, and DTs, to investigate large suicide outcome datasets. Reviews of the ML and suicide prevention literature indicate that methods vary substantially across studies and range significantly in rigor and model testing, with retrospective studies more represented than prospective studies and supervised learning models generally used more frequently than unsupervised learning techniques [20]. Overall, AI and ML have emerged as promising ways to improve the detection of suicide risk [20,21,26].

Recent years have seen an increase in the use of EMA data collection methods in suicide research, often described in quantitative case series studies [27], as part of randomized controlled trials of mobile apps [28], and in the investigation of suicide theories [29]. Therefore, an assessment of current research using these methods with such data are timely. In addition, previous research showed wide variability in the design and reporting of EMA studies assessing mental health and called for unifying standards for EMA reporting in mental health research going forward [30]. An emerging body of EMA datasets involving suicide outcomes and related factors holds the potential to substantially improve our understanding of complex suicide processes. Leveraging such unique datasets by applying AI strategies may offer a pivotal opportunity to advance our theoretical understanding and enhance prediction. Prediction can only support suicide prevention if it informs scalable, accessible, and evidence-based interventions.

EMA and digital phenotyping approaches have evolved and informed the development of ecological momentary interventions (EMIs), whereby treatments “are provided to people during their everyday lives (ie, in real time) and in natural settings” [31]. Research evidence on the effectiveness of EMI in the context of suicide prevention is scarce despite advances in the technology used to operationalize it. In a review of EMIs in suicide prevention research, Jiménez-Muñoz et al [12] reported that many of the available interventions had not yet been clinically tested, but those that had been tested (10 studies) showed good rates of effectiveness and feasibility. The most widely used intervention model in EMI studies in suicide research is the safety plan, which consists of strategies to help individuals during a suicidal crisis.

Relatedly, Coppersmith et al [32] argued that just-in-time adaptive interventions (JITAIs) hold the potential for reducing suicide rates. They are described as “an intervention design that adapts the provision of support (eg, the type, timing, and intensity) over time to an individual’s changing status and contexts, with the goal to deliver support at the moment and in the context that the person needs it most and is most likely to be receptive” [33]. Just-in-time adaptive interventions are used, particularly in health behavior change research, to address the states of vulnerability or periods of heightened susceptibility to negative health outcomes, such as smoking, unhealthy eating, and heavy drinking [34,35]. The emergence of a vulnerable state is seen as a dynamic process in which stable influences (eg, marital status and gender) and transient influences (eg, suicidal ideation and access to means) interact. They aim to capitalize on states of opportunity, namely periods of heightened susceptibility to positive health behavior changes (eg, healthy eating and physical activity), and have been put forward as feasible where states of vulnerability or opportunity emerge rapidly, unexpectedly, and ecologically (ie, in the individual’s natural environment). Just-in-time adaptive interventions involve 4 key features: decision points, intervention options, tailoring variables, and decision rules. Bryan et al [36] argued that further research is needed before such interventions can be constructed and applied as suicide prevention interventions, for example, knowing at an individual level when an intervention would be most effectively delivered (decision points), what that intervention should be (intervention options), the source and amount of the intervention to be delivered and in what context (tailoring variables), and what decision rules will inform when to offer which intervention. Therefore, further investigation of the application of AI to EMA datasets examining suicide outcomes is a necessary step to inform this next generation of digital interventions in suicide prevention. To our knowledge, this is the first systematic review that aims to synthesize empirical research on the application of AI to EMA data in the study of suicidal thoughts and behaviors.

The objectives of this review are to (1) synthesize published research on the application of AI strategies to EMA data in the study of suicidal ideation and behaviors; (2) identify the methodologies, study type and quality, EMA data collection procedures, suicide outcomes studied, types of AI strategies applied, and results reported; and (3) develop an adapted framework for reporting AI applied to EMA data in mental health research.

This review followed the recommendations of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta‐Analyses) guidelines [37]. The protocol has been preregistered on both PROSPERO (CRD42023440218) [38] and the Open Science Framework [39]. An account of changes to study protocol from preregistration is provided in Multimedia Appendix 1.

Studies were required to meet the following inclusion criteria, which are summarized in Textbox 1: (1) original, empirical research published in a peer-reviewed journal; (2) evaluated specific suicide outcomes—suicidal ideation, self-harm, nonsuicidal self-injurious behavior, suicide attempt, death by suicide; (3) used EMA to investigate suicide outcomes or related risk factors; (4) applied AI to EMA data; and (5) written in English. Interventional studies, observational studies, case series, and case reports were all included because such methodologies are suited to longitudinal EMA data collection. “Gray literature” and conference abstracts were also searched. Qualitative studies, protocols, opinion pieces, unpublished theses, and editorials were not included. Specifically, the types of EMA data collection included were (1) diaries, (2) experience sampling, (3) event-based sampling, and (4) passive sensing.

The following databases were searched for published studies: PsycINFO, PubMed, Scopus, and Embase. Search dates were as follows: initial search—July 20, 2023 (inception to July 2023); additional search—August 15, 2023 (inception to August 2023); final search before submission—June 2, 2024 (inception to June 2, 2024). Keywords were based on the following 3 fields: suicide, AI, and EMA. A search strategy, adapted for each database, was built using the Boolean operators “AND” and “OR” and applied to titles and abstracts. Three groups of terms were combined: the first relevant to suicide outcomes, the second relevant to EMA, and the third relevant to the application of AI or ML strategies. For example, search terms on Embase were as follows: suicide (suicide* / suicide ideation / suicidal / suicide attempt / suicide death) AND AI (Artificial Intelligence / AI / Machine Learning / ML) AND EMA (EMA / Ecological Momentary Assessment / Experience Sampling / Ambulatory Assessment).

To limit potential selection bias, we did not apply any restriction in terms of geographical location or population, but this was screened and extracted by researchers following the initial search. Studies not written in English were excluded. Details of the search strategy are provided in Multimedia Appendix 2.

Two researchers (RM and KMS) independently reviewed 10% (13/127) of the titles and abstracts, and 90% agreement was reached before the remaining 90% (114/127) of titles and abstracts were screened based on the inclusion criteria. The same process was repeated when reviewing the articles at the full-text stage. A third researcher was requested to make the final decision if consensus was not achieved. Citations were imported into EndNote (Clarivate), and duplicates were removed before being uploaded onto Rayyan (Rayyan Systems Inc) [40]. In addition, a manual search of reference lists from relevant published systematic reviews was conducted. Web of Science was used to undertake forward and backward citation searching of reference lists from included studies. On the basis of PROSPERO good practice guidance, searches were rerun just before the final analyses, and any further studies were identified and retrieved for inclusion. As high agreement was reached in the first screening (>90%), RM independently conducted the updated search and consulted with a second author (KMS) for any articles where there was uncertainty.

Data were extracted using a predefined form, and the data extraction process followed the procedures reported by published reviews conducted in this area [20,21]. Specifically, data were extracted independently by one rater, and 30% of the data were double checked by another rater; as a high rate of agreement was reached, the first author extracted the remaining 70% of the data. The authors were not blinded to information such as study author, institution, or journal. Authors were contacted if key information could not be ascertained from the article, its supporting information materials, or a previous publication referenced in the article that listed more details of the sample characteristics and study procedure. Characteristics of included studies were summarized and included the information presented in Textbox 2, wherever available.

Study outcomes are presented in a narrative synthesis. We descriptively reported on the study characteristics, study type, EMA data collection procedures used, study context (country and setting), study sample, device used, AI applied, and results reported. As a meta-analysis was not possible due to the small number of studies eligible for inclusion and the heterogeneity of outcomes reported, we presented the results reported in each published report. Where sufficient data were available, we presented an overview of the results reported by providing ranges and means. Study limitations were described narratively in published studies and were included in data extraction. Finally, new categories, in addition to those included in the CREMAS (adapted STROBE [Strengthening the Reporting of Observational Studies in Epidemiology] Checklist for Reporting Ecological Momentary Assessment Studies), were added, following discussion, to inform the development of a reporting framework.

Quality appraisal was conducted using CREMAS [41]. CREMAS identifies 16 items recommended to report on in EMA studies, which raters used for quality appraisal. A description of each item is available in Multimedia Appendix 3. The reporting checklist includes methodological features, such as EMA prompt strategy, monitoring periods, response latency, prompt frequency per day and interval, the type of technology used and its administration, alongside study design features to address potential sources of bias. The quality indicators were coded by one rater, with 30% double checked by a second rater. Discrepancies were resolved through discussion with a third author. As a high level of agreement was observed across raters, the first author completed coding on the remaining 70% of the quality indicator. As each criterion referred to a different aspect of study quality, the extent to which each study reported each of the 16 items is presented in the Results section.

This study did not require ethics approval as it summarized data from previously published studies.

A total of 1201 records were identified across the following databases: PubMed (n=655, 54.54%), Embase (n=517, 43.05%), Scopus (n=23, 1.92%), and PsycINFO (n=6, 0.5%). Additional records were identified through registers (n=48). Of the total 1249 records identified, duplicates were removed (n=786, 62.93%), records were marked as ineligible by automation tools (n=165, 13.21%), and some were removed for other reasons (n=171, 13.69%), such as not being written in English. The remaining 127 records were screened (title and abstract), and 58 (45.7%) of these were excluded. In total, 69 full-text records were retrieved for review and assessment of eligibility. A PRISMA flowchart (Figure 1) was created to graphically depict the inclusion and exclusion of studies in the final review, together with a PRISMA checklist (Multimedia Appendix 4). Of the 69 full-text records retrieved, 58 (84%) were excluded. The 58 articles were excluded for the following reasons: report described a study protocol (n=9, 16%), reported only qualitative data (n=4, 7%), reported no suicide outcome data (n=23, 40%), reported no EMA data collection (n=13, 22%), and the application of AI was not reported (n=9, 16%).

In parallel, additional 9 records were identified through websites (n=4, 44%) and citation searches (n=5, 56%), but none were included in the review following screening and eligibility assessment. A total of 12 studies were included in this review, indicating that this is a small but rapidly growing area of research. A descriptive overview of included studies is provided in Table 1.

Overall, a high level of heterogeneity was observed across the included studies in terms of meeting the reporting standards (Table 2). Studies met between 4 and 13 of the recommended 16 items, with an average of 7 items met across studies. Studies performed poorly in their reporting of item 3 (“description of EMA training procedures for participants”) and item 14 (“report on the treatment of missing data”). Studies tended to consistently meet item 5 (“state the number of waves in the study”), item 7 (“indicate the EMA prompting strategy used”), and item 8 (“state the intended frequency of prompts per day”).

The application of AI to EMA data in suicide research is an emerging and rapidly growing area of research. To our knowledge, this is the first systematic review to synthesize findings on the application of AI to EMA data in the study of suicidal ideation and behaviors. The aim of this review was to synthesize published research on the application of AI strategies to EMA data in the study of suicide to identify the methodologies used, AI strategies applied, and results reported and to provide an adapted CREMAS framework for reporting AI applied to EMA data in mental health research. Of the 12 studies included in this review, most (n=8, 67%) investigated suicidal ideation among clinical populations, and high heterogeneity was found with respect to the reporting of study procedures, particularly EMA data collection procedures. When quality was appraised across the 16 recommended items put forward by the CREMAS of reporting standards, studies met between 4 and 13 of the recommended 16 items, with an average of 7 items met across studies. Studies performed poorly in their reporting of item 3, “description of EMA training procedures for participants,” and item 14, “treatment of missing data.” These results are consistent with the findings reported by Davanzo et al [30], who investigated EMA in BPD specifically as part of a systematic review and meta-analysis.

Reflective of the broad use of EMA data in suicide research, standardized reporting procedures could allow for more direct comparisons and benefit the overall advancement of the field. The development of a refined reporting framework (Multimedia Appendix 5) by the research team aims to support transparency and consistency in this area going forward. CREMAS was designed to facilitate greater standardization of reporting in EMA studies broadly and was initially applied in diet and activity research [41]. The framework designed following this review expands on this by including items specific to the reporting of AI procedures and results (items 21 and 22) and reporting of mental health outcomes specifically (items 4 and 5). In addition, inconsistencies in reporting, highlighted in previous reviews of both EMA and AI, informed the addition of more detailed items to better capture EMA data collection procedures (item 7). One item was added to capture the diversity of data sources in such research, that is, clinical files, sensor data, self-reports, etc. Finally, inconsistency in reporting across studies in this review informed the addition of specific items, for example, in relation to how AI results are reported, to facilitate future reviewers in synthesizing data [21,22].

In terms of AI strategies used, studies used CART [43,50], RF algorithms [44,49], elastic net models [44,46-49,51-53], and RNNs [45,51]. The best-performing models across accuracy, sensitivity, specificity, and PPV were multilevel CART [50] and elastic net models [53]. Previous research by Tang et al [54] investigating the effectiveness of explainable AI in suicide risk assessment indicated that DT, RF, and extreme gradient boosting models achieved the best results, while DT had the best performance with an AUC of 0.95 in non-EMA suicide datasets. In the application of AI to EMA data to predict suicidal ideation in the short term, studies reported mean AUCs (0.736-0.86), sensitivity (0.64-0.81), specificity (0.73-0.86), and PPVs (0.72-0.77). Studies that compared the accuracy of models found that the best-performing models required self-reported information derived from EMAs, whereas sensor-based data added negligible predictive accuracy. However, study authors noted that sensor-based assessments for sleep activity and heart rate were measured with emerging wearable sensor technology that may introduce more error than standard laboratory assessment [49].

Another notable finding is the importance of variability in prediction. In patients who were acutely suicidal, variability in suicidal thinking provided more information about suicide risk than average levels of suicidal thinking [26,53]. Wang et al [53] found that collecting real-time data about suicidal thinking in the course of a hospital stay significantly improved short-term prediction of posthospitalization suicide attempts. Models that included dynamic changes in suicidal thinking over time yielded the best prediction; features that captured rapid changes in suicidal thoughts were particularly strong predictors. Similarly, Bonilla-Escribano et al [49] reported that patients who were suicidal were best clustered into 2 groups with EMA data: low and high variability. The high-variability group showed more instability in all dimensions, particularly in social withdrawal, sleep measures, wish to live, and social support. Both clusters were separated by 10 clinical features (AUC=0.74), including depressive symptoms, cognitive instability, the intensity and frequency of passive suicidal ideation, and the occurrence of clinical events, such as suicide attempts or emergency visits during follow-up.

Findings are consistent with previous research, indicating that suicidal ideation severity varies considerably from hour to hour [55]. Survey noncompletion also emerged as an important predictor of posthospitalization suicide attempts [53]. Clinical initiatives to follow up with patients who are suicidal with ecological measures should take into account the existence of this high variability cluster and survey noncompletion to inform targeted support.

Researchers also examined the extent to which the EMA data type added to the predictive capabilities of a model. Peis et al [51] found that the addition of EMA records boosted the system recall to predict a suicidal ideation diagnosis from 48.13% obtained exclusively from EHR-based methods to 67.78%. Czyz et al [47] reported that the best-performing models in their study required self-reported information derived from EMAs, whereas sensor-based data had negligible predictive accuracy. The findings reported may also be reflective of the specific variables being measured using sensors. For example, previous research by Liu et al [56] achieved classification accuracy ranging from 96% to 98% for 3 risk levels across different modeling methods by applying ML to eye and head signal data. They identified that high-risk individuals experience psychomotor retardation and symptoms of anxiety and depression characterized by eye contact avoidance, slower blinks, and a downward eye gaze.

A common criticism of ML applied to EHR datasets retrospectively in health care research is that it is not known how models would perform if tested prospectively on truly independent patient samples. For example, in a non-EMA study [57], ML models predicted treatment outcomes among patients with schizophrenia with high accuracy within the trial in which the model was developed but performed no better than chance when applied out of sample. Researchers point to the challenge inherent in applying a model developed specifically in relation to one dataset to another clinical sample or setting. Salganik et al [58] measured the predictability of life outcomes broadly in a scientific mass collaboration. Despite using a rich dataset and applying ML methods optimized for prediction, the best predictions were not very accurate and only slightly better than those from a simple benchmark model. One important advantage of this review is that it reports on EMA data from predominantly prospective studies carried out in clinical settings with models identified a priori. EMA data collection procedures themselves may help to address challenges with EHR and other clinic-based data by reducing recall bias and capturing high levels of variability at an individual level.

While this is a growing area of research, a small number of published studies were identified and included in this review. Therefore, results should be interpreted with appropriate caution. Heterogeneity across data collection procedures, data analysis, and reporting of results hampered comparison. The authors argue that this may be understood in the context of a lack of standardized reporting guidelines, which motivated this work. AI and suicide research can also be dispersed across disciplines, reflecting diverse emphases revealing areas of expertise and research focus. Graham et al [59] argued that a diverse community of experts vested in mental health research and care, including scientists, clinicians, regulators, and patients, must communicate and collaborate to realize the full potential of AI. Although not an explicit aim of this review, the diversity of data collection procedures reported and the limited quantitative data available in general meant that it was not possible to compare groups. Further published research in this area would facilitate the completion of a meta-analysis to synthesize data on AI applied to EMA in predicting suicide outcomes across accuracy, sensitivity, specificity, and PPV and negative predictive value.

In terms of outcomes, most (7/12, 70%) studies reported on the ability of AI applied to EMA data to predict suicidal ideation rather than behavior. In addition, heterogeneity was noted in both the measurement and administration of suicidal ideation assessment measures, with some (2/12, 17%) studies relying on a single item of suicidal ideation for assessment [42,44]. There is a need for researchers to use established standardized guidelines when reporting model performance, such as the guidelines for developing and reporting ML predictive models in biomedical research [60].

Sample characteristics also limit the generalization of results reported outside of the specific clinical groups and settings in which studies were conducted. Authors reported that most participants were women [43,46], in some cases over 85% [46], and predominantly White [46,53]. Studies rarely included a control group, and the homogenous samples used (eg, patients with BPD, gender and sexual minority individuals, and patients who are admitted to the hospital for psychiatric care) while providing rich data regarding vulnerable groups limit generalization across clinical samples and settings.

Efforts to overcome heterogeneity in data collection and reporting procedures are needed to advance research in this area. The recent prioritization of open science and archival data sharing, together with greater integration of EHR in health systems and the proliferation of devices that facilitate EMA data collection, provides greater opportunities to apply innovative ML strategies to large mental health datasets. However, standardization of procedures and reporting is required to leverage such datasets to inform our understanding of suicide processes and develop targeted supports. The development of an adapted framework by the research team for reporting on studies applying AI strategies to EMA data in mental health research (Multimedia Appendix 5) aims to address this.

Future research should aim to validate clinical prediction models in different clinical samples. Chekroud et al [57] noted that this generally results in predictive performance measures that are lower but allows a more realistic indication of the potential of the model to inform clinical practice [61]. For AI, and in particular ML, to enhance clinical practice, the models developed must robustly predict outcomes for unseen, future patients. The generalizability of ML models beyond the data that were used to develop them is crucial for the clinical success of model efforts. In many cases, ML models have been evaluated in small samples of highly selected participants, and little is known about their potential for predicting mental health in other cohorts and settings [61]. Consequently, researchers have begun to depart from traditional participant pools toward more population-reflective, crowdsourced data collection [62]. Such big data approaches, recruiting participants worldwide and from diverse demographic backgrounds, may offer more representative data. Another key advantage of mobile assessment platforms is that they are more amenable to repeated and triggered assessments [61]. Further prospective EMA studies provide an opportunity to overcome many of the challenges inherent in generalizing models developed from EHR, clinical trial, or cross-sectional datasets.

Although EMA-based sampling strategies seem uniquely suited to capturing meaningful variation in suicidal thoughts and behaviors [63], there is uncertainty regarding how best to balance intensity and duration of assessment to capture this variation. Episodes of suicidal ideation have been found to be brief, with participants reporting most episodes to be shorter than an hour [64]. If the process assessed is faster than the assessment interval, EMA data and associations derived may not be inherently more valuable, particularly when measurements are not evenly spaced in time (eg, when EMA assessments are randomized throughout the day). The choice of an assessment and data analytic approach therefore requires theoretical justification and should be based on estimates of how long episodes of suicidal thinking may last. Future research in this area should systematically examine theoretically informed real-time sampling and modeling strategies of suicidal ideation.

RNNs, as found in this review [45,51,52], are potentially important tools for establishing model-free predictor importance. RNNs and other “black box” models have shown good prediction accuracy, but it can be difficult to explain the prediction made. A better understanding of how models operate can enable the detection of bias and faults of the model that can arise through biased training sets. Predictions from unexplainable models also pose substantial challenges to implementation, as the uptake of model predictors depends strongly on clinicians understanding and trusting them [62]. Evaluation of the importance of individual predictors in the context of highly correlated data presents challenges requiring appropriate methodology. Explainable predictive models, such as the Shapley Additive Explanatory approach, may be helpful in predicting and understanding the importance of features for suicide outcomes. Further research has shown that ML models with the Shapley Additive Explanatory approach are able to interpret and understand the nature of an individual’s predictions of suicidal behavior [65]. Future research using explainable predictive models may offer the opportunity to harness the predictive ability of ML models while providing transparency to predictions made. Clinically, such developments could inform personalized assessment and risk formulation and guide the development of clinical decision support tools.

The findings of this review indicate that the application of AI to EMA data within suicide research is a small but burgeoning area. Substantial differences were apparent in reporting standards across studies, particularly with respect to EMA data collection procedures. Results indicate the strength of ML applied to self-report EMA data in the prediction of near-term suicidal ideation. The development of an adapted CREMAS reporting framework (Multimedia Appendix 6) by the research team aims to standardize reporting on the application of AI to EMA data in mental health research going forward.

An important goal of research in this area is to (1) identify models that will help predict who will experience suicide outcomes (necessary to inform clinical decision support tools) and (2) identify what supports work for whom and in what time frame (necessary to inform EMIs or just-in-time adaptive interventions). To achieve these goals, further prospective EMA studies applying ML models to diverse clinical samples, using standardized data collection and reporting procedures, are recommended. The application of AI to EMA data within suicide research offers the opportunity to move beyond a focus on risk classification toward dynamic data-driven risk formulation, consistent with recent clinical practice guidance [65]. In addition, integrating explainable AI could offer greater transparency to better inform clinician assessment and formulation and ultimately guide data-driven, personalized, and effective supports.