Ecological Momentary Assessment and mHealth Interventions Among Men Who Have Sex With Men: Scoping Review

Background: Ecological momentary assessment (EMA) is a research design that allows for the measurement of nearly instantaneous experiences within the participant’s natural environment. Using EMA can help improve recall bias, ecological validity, and patient engagement while enhancing personalization and the ubiquity of interventions. People that can benefit from the use of EMA are men who have sex with men (MSM). Previous EMA studies have been successful in capturing patterns of depression, anxiety, substance use, and risky sexual behavior. These findings are directly relevant to MSM, who have high rates of each of these psychological and behavioral outcomes. Although there is a driving force behind the growing literature surrounding EMAs among MSM, no synthesizing reviews yet exist. Objective: The aims of this study were to (1) synthesize the literature across fields on how EMA methods have been used among MSM, (2) better understand the feasibility and acceptability of EMA interventions among MSM, and (3) inform designs for future research studies on best evidence-based practices for EMA interventions. Methods: Based on 4 library databases, we conducted a scoping review of EMAs used within interventions among MSM. The eligibility criteria included peer-reviewed studies conducted in the United States and the use of EMA methodology in an intervention for MSM. Modeling after the Centers for Disease Control and Prevention’s Compendium of Evidence-Based Interventions as the framework, we applied a typology that used 8 distinct review criteria, for example, sample size, design of the intervention, random assignment, design of the follow-up investigation, rate of retention, and rate of engagement. Results: Our results (k=15, N=952) indicated a range of sample sizes; the smallest sample size was 12, while the largest sample size was 120. Of the 15 studies, 7 (47%) focused on outcomes related to substance use or outcomes related to psychological experiences. Of the 15 studies, 5 (33%) implemented an EMA intervention across 30 days. Of the 15 studies, 2 studies (13%) used random assignment, and 2 studies (13%) had quasi-experimental designs. Of the 15 studies, 10 studies (67%) reported acceptable retention rates greater than 70%. The outcomes that had event-contingent prompts (ie, prompts after engaging in substance use) were not as effective in engaging participants, with overall engagement rates as low as 37%. Conclusions: Our systematic scoping review indicates strong evidence that the EMA methodology is both feasible and acceptable at high rates among MSM, especially, when examining psychological and behavioral outcomes such as negative or positive affect, risky sexual behavior, or substance use. Further research on optimal designs of EMA interventions for MSM is warranted. (J Med Internet Res 2021;23(8):e27751) doi: 10.2196/27751


Ecological Momentary Assessment
Developed originally from the field of personality and social psychology, ecological momentary assessment (EMA) is a research design with methodological components that allow researchers to measure experiences as close to that moment as possible and within the participant's natural environment [1]. Methodological strategies in EMAs have included prompting participants at various time intervals using self-report surveys or triggers specific to locations, events, or both [2]. Researchers have indicated that recall-based self-reports can be inaccurate or unreliable measurements of participants' actual lived experiences [2]. The goal of EMA is to minimize retrospective recall issues and enhance researchers' ability to measure lived experiences of people in the moment [1].
The ability to measure and potentially intervene in lived experiences in the moment is especially important to impact dysfunctional thoughts, capture psychological distress, or even intervene in harmful behavior [2]. In addition, one of the key benefits and goals of EMA is to provide high levels of ecological validity, or validity that comes from collecting data and implementing an experiment in a participant's natural setting in real-world contexts. High ecological validity can enhance the ability for research findings to be applied to real-world scenarios, increasing the likelihood of generalizability [3]. Research has also found EMA methods to outperform traditional paper-pencil measurements in the ability to determine needs of clinical interventions more precisely. A primary reason that EMA measurements outperform traditional paper-pencil measurements is that repeated measurements minimize the effect of participants' current state on results [4]. Finally, technology-based interventions incorporating EMA methods have shown promise in terms of feasibility and acceptability of enhancing intervention outcomes [5].

Men Who Have Sex With Men
Men who have sex with men (MSM) have been found to show high rates of both psychological distress and engagement in various risky behaviors [6]. Specifically, studies have found MSM to endure higher levels of depression, anxiety, substance use, and risk of contracting HIV [7,8]. EMAs have been used among MSM in daily diaries since 2007 [7] and have evolved tremendously into the realm of internet use [9], smartphone technology [10], and interventions [11]. Use of EMAs among MSM is a growing area of research. EMAs have been shown to be highly effective in reaching people who have a history of substance use or other risky behavior, due to the minimization of stigma and enhancement of self-control over privacy, confidentiality, and anonymity [10,12,13].

Scoping Review
The primary purpose of a scoping review is to synthesize current literature surrounding a topic area. Thus, the synthesis produced from a scoping review acts as a summary of available literature, a means to identify key concepts, and a precursor to a systematic review [14]. To the best of the authors' knowledge, neither a scoping review nor a systematic review has yet been produced on the topic of EMA use among MSM, due to the limited literature surrounding the topic. As a result, the authors intended to conduct a scoping review by applying the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Extension for Scoping Reviews) [15], as a contribution toward a future systematic review based on an increase in literature. Authors also acknowledge the limitations of a scoping review, mainly, an inability to make quality assurances, a lack of strong validity, and an inability to hypothesize based on the review.

Scoping Review for EMAs Among MSM
Despite the benefits and clinical implications of using EMA methods for at-risk populations, there have been no reviews compiling the literature of how EMAs have been used among MSM. One growing method for synthesizing theoretical and empirical evidence in the literature is the scoping review [16]. Scoping reviews are considered as means to describe key findings across literature, identify gaps in research, and inform the design of future research studies [16]. Two major benefits of conducting a scoping review are the ability to examine the breadth of the topic of EMA methodologies that are applied to MSM, specifically, within interventions, and the ability to identify knowledge gaps and future directions for the expansion of this area of research. The aims of this study were to: (1) synthesize the literature across fields on how EMA methods have been used among MSM, (2) better understand the feasibility and acceptability of EMA interventions among MSM, and (3) to inform designs for future research studies on best evidence-based practices for EMA interventions.

Eligibility Criteria
We identified several eligibility criteria that needed to be met for a publication to be included in this scoping review. The eligibility criteria included MSM samples; EMA interventions or assessments or determinations of EMA's efficacy, acceptability, and feasibility; a publication date within the past 5 years; peer-reviewed studies conducted in the United States; and quantitative data analyses.

Information Sources
We chose 4 prominent databases to retain studies from: Ovid Medline, which focuses on biomedical scholarly literature; Psychological and Behavioral Science Collection, which focuses on mental processes and emotional and behavioral experiences; PsycInfo, which focuses on behavioral and social science research; and Cumulative Index to Nursing and Allied Health Literature, which provides access to health research, specifically, in nursing and other allied health. EMAs have predominately been applied to behavioral or psychological health [17,18] and physical health [19,20]. Therefore, these databases were determined as most relevant and applicable to the specific topic of this scoping review ( Figure 1).

Keywords and Search Process
To ensure that we conducted an inclusive review of the literature across databases, we used multiple search terms for our target populations.
Step 1 involved searching the terms "homosexuality" and "homosexual." In step 2, we used the search term "sexual minority men." Finally, in step 3, we used the search terms "men who have sex with men" and "MSM." In step 4, we combined all these terms to achieve the most comprehensive review of literature pertaining to our target population of MSM. In step 5, we added the keywords "ecological momentary assessment" or "EMA," "daily diaries," and "experience sampling" to identify all EMA-related literature from these databases. In step 6, we combined our MSM search terms with the EMA terms to narrow down to only the most directly relevant articles. Upon reaching step 7, we applied the term "intervention" to further narrow the articles included in this review.
Step 8 was comprised of combining all search terms

Inclusion Criteria, Exclusion Criteria, and the Iterative Process
Once we had the initial studies from our database searches, we combined all the study titles and previewed the articles. Articles were further narrowed based on 2 additional inclusion criteria and 2 additional exclusion criteria; studies were excluded if they were not published in the past 5 years and if they were duplicative across databases. Studies were included if they were conducted in the United States and included quantitative data and analyses ( Figure 1).
As suggested by the clarity of guidelines in scoping reviews [21], our search and review were conducted in an iterative manner over time. We conducted our first search in January 2020, our second search in May 2020, our third search in September 2020, and the last search in March 2021, as presented, to examine quarterly changes in the literature.

Analysis
Modeling after the Centers for Disease Control and Prevention (CDC) Compendium of Evidence-Based Interventions (EBIs) [22], we primarily used 8 distinct criteria to review the final set of eligible publications [9,11,13,[23][24][25][26][27][28][29][30][31][32][33][34]. The EBI criteria and best practices in the compendium, developed and defined by the CDC's Prevention Research Synthesis project, posit a series of systematic review components for interventions. EBI criteria have been shown to generate significant effects and strong evidence of efficacy in HIV-related outcomes [22]. Based on the compendium of EBIs, our review criteria included (1) citation, sample size, and duration of study; (2) location; (3) random assignment (yes or no); (4) key aspects of the intervention; (5) follow-up (yes or no); (6) occurrence of follow-up after intervention; (7) rate of retention; and (8) rate of engagement (Multimedia Appendix 1). We also conducted a secondary analysis comprised of additional review criteria: recruitment strategy, description of intervention and comparison of arms, specific measurement tools, outcomes of interest, and outcome results (Multimedia Appendix 2).

Risk of Quality in Individual Studies
Using the CDC compendium of best evidence-based risk reduction for individual-level interventions, we evaluated the strength of each study. None of the studies reviewed met full criteria for best standards of risk reduction in individual-level interventions. However, we continued evaluating based off CDC compendium criteria to determine study designs, intervention elements, and highest standards currently achieved.

Study Selection
A total of 129 articles were identified in the preliminary search: 24 from the Psychology and Behavioral Science Collection, 5 from OVID Medline, 58 from PsycInfo, and 42 from Cumulative Index to Nursing and Allied Health Literature. Among these 129 articles, we excluded 114 studies for not meeting additional screening criteria. These 4 criteria that the studies had to meet were: (1) the study was published in the past 5 years, eliminating 44 studies; (2) not a duplicative study, eliminating 38 studies; (3) a US-based study, eliminating 12 studies; and (4) the study used a quantitative analysis, eliminating 9 studies. The qualifying criteria led to a final set of 15 studies in this review [9,11,13,[23][24][25][26][27][28][29][30][31][32][33][34].

Random Assignment
Random assignment occurred in 2 of the 15 studies (13%) included in our review [11,26]. The CDC recommends random assignment as a gold standard, to rule out biases in a systematic way across multiple intervention arms. Since the majority of the studies (11/15, 73%) included in this review had only 1 arm, random assignment was not implemented [9,13,[23][24][25]28,29,[32][33][34]. In the remaining 2 studies that had nonrandomized designs with multiple arms, one study used a quasi-experimental design implementing clinical cutoffs for hypersexuality to determine group membership [30], while the other study assigned groups based on whether participants were recruited in-person or online [9]. Random assignment was considered as potentially unethical in many of these studies due to their focus on substance use and sexual behavior; thus, a quasi-experimental design was better suited.

Follow-up and Occurrence of Follow-up
Follow-up assessments were administered in 2 studies of the 15 (13%) [11,31], one of which included follow-ups at 3 different time points: 60 days, 90 days, and 120 days [31], while the other followed up with participants after 4 weeks [11]. According to the CDC compendium, there should be a follow-up, and it should, specifically, occur more than 30 days after completion of the intervention [22].

Rate of Engagement
Of the 15 studies, 13 (87%) reported rate of engagement [9,13,[23][24][25][26][27][28][29][31][32][33][34]. Engagement rate was defined as an overall rate of completion for text or online surveys, text prompts, or daily diaries, depending on the study modality and was reported consistently across all studies. The average overall engagement rate was 76.93%, the lowest overall engagement rate was 37.3% [9], and the highest was 98.7% [13]. Engagement is a key component of retention, and the 70% retention rate is recommended as the benchmark for an acceptable engagement rate [22].

Outcome Results
Intervention studies using EMA methods have demonstrated success in longitudinally measuring substance use, compared with studies that relied on non-EMA measurements such as timeline follow-back surveys [9,23,32]. Additionally, EMA methods generated greater acceptability than other methods: daily diaries had high rates of response completion. The highest response rate was 97.3% [9,34], and the lowest was only 84% [23].

Principal Findings
In our scoping review, we aimed to provide an overview of the growing literature on a relatively novel measurement: ecological momentary assessment (EMA). We found that among men who have sex with men (MSM), the majority of EMAs have been used to intervene on risk-taking behaviors such as alcohol and drug use or unprotected sex with multiple partners. Although risk-taking behaviors have often been stigmatized, the use of EMA through smartphone technology has been seen as a highly effective way to safely assess risk-taking behaviors [10,12,13]. Overall, EMA was seen as an acceptable and feasible method, with daily diaries as the most acceptable tool [9,23,[32][33][34] to collect the experiences of MSM. A unique facet of MSM research was the successful use of recruitment strategies beyond technology-based recruitment, which included assistance in initiation and engagement from community-based organizations [11,[25][26][27]30,31]. This scoping review was used as a synthesizing method with a wide array of review dimensions and criteria such as quantitative interventions among MSM. This allowed our results to provide a comprehensive set of typological frameworks that may be useful in designing and implementing an EMA-integrated intervention for behavioral change. Basing typographic dimensions off preexisting frameworks offered by the CDC [22], we incorporated the most salient components for intervention research. This allowed for a better assessment of the strength of existing EMA interventions among MSM. Also, we conducted 4 time points of literature searches (ie, January 2020, June 2020, September 2020, and March 2021), for inclusion of more studies, which increased the comprehensiveness of this scoping review.
Our study determined that there were limited psychometrically sound EMA measurements that were fully validated. Given the growing research attention on EMAs within the context of behavioral intervention, future studies may aim to develop and validate EMA measurements. EMA has been widely used as a just-in-time assessment and monitoring tool, but it also can be a great measurement resource to predict behavioral outcomes. We suggest future research should focus on developing predictive models and analytic methods, using intensive longitudinal data from EMAs to understand behavioral changes or outcomes over time. In terms of an analytic perspective, since EMAs lead to extensive longitudinal data, the risk of missingness and the handling of missing data will become more prevalent. Therefore, studies on appropriate analytic approaches to manage missing data from EMAs will be essential.
Although EMAs may reduce recall bias, due to the repetitiveness of measurement, they can also increase participant bias and burden [35,36]. Future research should take into consideration EMA designs that are engaging but protective of data anonymity and confidentiality, to prevent participant biases such as the social desirability effect and the halo or devil effect. To avoid priming of such participant biases, details and information in EMA-based interventions should be presented in a judgment-free manner. In order to reduce psychological reactions, future research should consider developing EMAs as self-motivated mechanisms, with use options such as event-contingent prompts, daily diaries, text prompts, or other mechanisms. Additionally, future studies should examine putative mechanism factors such as resilience and social support, to develop a comprehensive, integrative intervention program for MSM [37].

Limitations
There were limitations imposed by the scope and design of the study. First, the inclusion of the major library databases focused on studies most relevant to the population and methodological strategy of interest, but we excluded other minor library databases. Therefore, a future direction may include an inclusion of minor databases in the review for EMA interventions among MSM. Second, many of the studies included in this review were feasibility and acceptability tests as well as pilot studies with inconsistent assessments of outcomes, thus minimizing the effectiveness of a meta-analysis or systematic review. Although we presented results from a scoping review to provide an overview and the state of EMA in behavioral medicine, future research may conduct a systematic review or meta-analysis, as the prevalence of empirical evidence from randomized controlled trials using EMAs in this area is likely to increase [38].

Conclusions
Leveraging evidence-based intervention designs with validated ecological momentary assessments can advance our understanding of factors and processes in behavioral changes and health outcomes. These approaches can be further empowered through technology-based behavioral medicine and social medicine. In this scoping review paper, we provided a typology of EMA-based intervention research that was designed to promote health behavior and psychological well-being. Advancements in psychometric tests to validate EMAs will be critical. As the empirical evidence and theories in this field are emerging, we hope our review offers some guidance and synthesis of the literature to develop and evaluate technology-based EMA health interventions.