This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Ecological momentary assessment (EMA) is a research design that allows for the measurement of nearly instantaneous experiences within the participant’s natural environment. Using EMA can help improve recall bias, ecological validity, and patient engagement while enhancing personalization and the ubiquity of interventions. People that can benefit from the use of EMA are men who have sex with men (MSM). Previous EMA studies have been successful in capturing patterns of depression, anxiety, substance use, and risky sexual behavior. These findings are directly relevant to MSM, who have high rates of each of these psychological and behavioral outcomes. Although there is a driving force behind the growing literature surrounding EMAs among MSM, no synthesizing reviews yet exist.
The aims of this study were to (1) synthesize the literature across fields on how EMA methods have been used among MSM, (2) better understand the feasibility and acceptability of EMA interventions among MSM, and (3) inform designs for future research studies on best evidence-based practices for EMA interventions.
Based on 4 library databases, we conducted a scoping review of EMAs used within interventions among MSM. The eligibility criteria included peer-reviewed studies conducted in the United States and the use of EMA methodology in an intervention for MSM. Modeling after the Centers for Disease Control and Prevention’s Compendium of Evidence-Based Interventions as the framework, we applied a typology that used 8 distinct review criteria, for example, sample size, design of the intervention, random assignment, design of the follow-up investigation, rate of retention, and rate of engagement.
Our results (k=15, N=952) indicated a range of sample sizes; the smallest sample size was 12, while the largest sample size was 120. Of the 15 studies, 7 (47%) focused on outcomes related to substance use or outcomes related to psychological experiences. Of the 15 studies, 5 (33%) implemented an EMA intervention across 30 days. Of the 15 studies, 2 studies (13%) used random assignment, and 2 studies (13%) had quasi-experimental designs. Of the 15 studies, 10 studies (67%) reported acceptable retention rates greater than 70%. The outcomes that had event-contingent prompts (ie, prompts after engaging in substance use) were not as effective in engaging participants, with overall engagement rates as low as 37%.
Our systematic scoping review indicates strong evidence that the EMA methodology is both feasible and acceptable at high rates among MSM, especially, when examining psychological and behavioral outcomes such as negative or positive affect, risky sexual behavior, or substance use. Further research on optimal designs of EMA interventions for MSM is warranted.
Developed originally from the field of personality and social psychology, ecological momentary assessment (EMA) is a research design with methodological components that allow researchers to measure experiences as close to that moment as possible and within the participant’s natural environment [
The ability to measure and potentially intervene in lived experiences in the moment is especially important to impact dysfunctional thoughts, capture psychological distress, or even intervene in harmful behavior [
Men who have sex with men (MSM) have been found to show high rates of both psychological distress and engagement in various risky behaviors [
The primary purpose of a scoping review is to synthesize current literature surrounding a topic area. Thus, the synthesis produced from a scoping review acts as a summary of available literature, a means to identify key concepts, and a precursor to a systematic review [
Despite the benefits and clinical implications of using EMA methods for at-risk populations, there have been no reviews compiling the literature of how EMAs have been used among MSM. One growing method for synthesizing theoretical and empirical evidence in the literature is the scoping review [
We identified several eligibility criteria that needed to be met for a publication to be included in this scoping review. The eligibility criteria included MSM samples; EMA interventions or assessments or determinations of EMA’s efficacy, acceptability, and feasibility; a publication date within the past 5 years; peer-reviewed studies conducted in the United States; and quantitative data analyses.
We chose 4 prominent databases to retain studies from: Ovid Medline, which focuses on biomedical scholarly literature; Psychological and Behavioral Science Collection, which focuses on mental processes and emotional and behavioral experiences; PsycInfo, which focuses on behavioral and social science research; and Cumulative Index to Nursing and Allied Health Literature, which provides access to health research, specifically, in nursing and other allied health. EMAs have predominately been applied to behavioral or psychological health [
Flow diagram of study selection process. CINAHL: Cumulative Index to Nursing and Allied Health Literature; *Peer-reviewed articles.
To ensure that we conducted an inclusive review of the literature across databases, we used multiple search terms for our target populations. Step 1 involved searching the terms “homosexuality” and
Once we had the initial studies from our database searches, we combined all the study titles and previewed the articles. Articles were further narrowed based on 2 additional inclusion criteria and 2 additional exclusion criteria; studies were excluded if they were not published in the past 5 years and if they were duplicative across databases. Studies were included if they were conducted in the United States and included quantitative data and analyses (
As suggested by the clarity of guidelines in scoping reviews [
Modeling after the Centers for Disease Control and Prevention (CDC) Compendium of Evidence-Based Interventions (EBIs) [
Using the CDC compendium of best evidence-based risk reduction for individual-level interventions, we evaluated the strength of each study. None of the studies reviewed met full criteria for best standards of risk reduction in individual-level interventions. However, we continued evaluating based off CDC compendium criteria to determine study designs, intervention elements, and highest standards currently achieved.
A total of 129 articles were identified in the preliminary search: 24 from the Psychology and Behavioral Science Collection, 5 from OVID Medline, 58 from PsycInfo, and 42 from Cumulative Index to Nursing and Allied Health Literature. Among these 129 articles, we excluded 114 studies for not meeting additional screening criteria. These 4 criteria that the studies had to meet were: (1) the study was published in the past 5 years, eliminating 44 studies; (2) not a duplicative study, eliminating 38 studies; (3) a US-based study, eliminating 12 studies; and (4) the study used a quantitative analysis, eliminating 9 studies. The qualifying criteria led to a final set of 15 studies in this review [
The average sample size across the 15 studies reviewed was 63.5 (SD 31.85). The CDC recommends that each sample is greater than 50 participants per study arm. Among the 15 studies reviewed, 10 (67%) had sample sizes >50, and 4 (27%) had sample sizes ≥100 [
The only 2 demographic measurements reported by all 15 articles were race/ethnicity and age [
From this review, we determined that multiple studies had similar key aspects across the interventions implemented. First, 7 of the 15 studies (47%) focused on multiple types of substance use, including nicotine use, alcohol use, or other substance use (eg, cocaine, methamphetamine, and cannabis). Second, affect and stigma were discussed as primary outcomes for 5 of 15 studies (33%) [
Random assignment occurred in 2 of the 15 studies (13%) included in our review [
Across the articles, there were a wide variety of intervention strategies used for implementation, for example, impacting the duration, mechanism for the intervention, and tasks for intervention participants. The average duration of the intervention was 31.5 days; the shortest intervention was a single-day cross-sectional survey [
Follow-up assessments were administered in 2 studies of the 15 (13%) [
Rate of retention was measured in 33% (5/15) of the included studies [
Of the 15 studies, 13 (87%) reported rate of engagement [
The location of the catchment areas and study sites varied. The majority of studies (9/15, 60%) were conducted on the East Coast [
Of the 15 studies, 12 (80%) used more than 2 recruitment strategies. Of those that used at least 2 recruitment strategies, 14/15 (93%) studies paired their strategies with social media (e.g., Instagram or Facebook) [
The majority of the studies (10/15, 67%) were conducted with EMA surveys [
There were 3 prominent outcomes of interest across the included studies: risky sexual behavior, substance use, and acceptability. Of the 15 studies, 7 (47%) measured substance use status, including use of nicotine, alcohol, and other nonprescription drugs [
Intervention studies using EMA methods have demonstrated success in longitudinally measuring substance use, compared with studies that relied on non-EMA measurements such as timeline follow-back surveys [
In our scoping review, we aimed to provide an overview of the growing literature on a relatively novel measurement: ecological momentary assessment (EMA). We found that among men who have sex with men (MSM), the majority of EMAs have been used to intervene on risk-taking behaviors such as alcohol and drug use or unprotected sex with multiple partners. Although risk-taking behaviors have often been stigmatized, the use of EMA through smartphone technology has been seen as a highly effective way to safely assess risk-taking behaviors [
Our study determined that there were limited psychometrically sound EMA measurements that were fully validated. Given the growing research attention on EMAs within the context of behavioral intervention, future studies may aim to develop and validate EMA measurements. EMA has been widely used as a just-in-time assessment and monitoring tool, but it also can be a great measurement resource to predict behavioral outcomes. We suggest future research should focus on developing predictive models and analytic methods, using intensive longitudinal data from EMAs to understand behavioral changes or outcomes over time. In terms of an analytic perspective, since EMAs lead to extensive longitudinal data, the risk of missingness and the handling of missing data will become more prevalent. Therefore, studies on appropriate analytic approaches to manage missing data from EMAs will be essential.
Although EMAs may reduce recall bias, due to the repetitiveness of measurement, they can also increase participant bias and burden [
There were limitations imposed by the scope and design of the study. First, the inclusion of the major library databases focused on studies most relevant to the population and methodological strategy of interest, but we excluded other minor library databases. Therefore, a future direction may include an inclusion of minor databases in the review for EMA interventions among MSM. Second, many of the studies included in this review were feasibility and acceptability tests as well as pilot studies with inconsistent assessments of outcomes, thus minimizing the effectiveness of a meta-analysis or systematic review. Although we presented results from a scoping review to provide an overview and the state of EMA in behavioral medicine, future research may conduct a systematic review or meta-analysis, as the prevalence of empirical evidence from randomized controlled trials using EMAs in this area is likely to increase [
Leveraging evidence-based intervention designs with validated ecological momentary assessments can advance our understanding of factors and processes in behavioral changes and health outcomes. These approaches can be further empowered through technology-based behavioral medicine and social medicine. In this scoping review paper, we provided a typology of EMA-based intervention research that was designed to promote health behavior and psychological well-being. Advancements in psychometric tests to validate EMAs will be critical. As the empirical evidence and theories in this field are emerging, we hope our review offers some guidance and synthesis of the literature to develop and evaluate technology-based EMA health interventions.
Primary scoping review analysis.
Secondary scoping review analysis.
Demographics.
Centers for Disease Control and Prevention
evidence-based intervention
ecological momentary assessment
men who have sex with men
Preferred Reporting Items for Systematic Reviews and Meta-Analyses - Extension for Scoping Reviews
VC conceived the study. VC and SJK were responsible for the study design and concept. SJK was responsible for accurate implementation of scoping review methodology. VC was responsible for the literature search and literature review. SJK supervised the study design and the review process. VC was responsible for constructing figures and multimedia appendices. VC and SJK collaborated on the discussion and study impact. VC and SJK drafted the first draft of the manuscript. All authors extensively reviewed and approved the manuscript before submission for peer review.
None declared.