This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Men who have sex with men (MSM) face disproportionate risks concerning HIV and other sexually transmitted infections, substance use, and mental health. These outcomes constitute an interacting syndemic among MSM; interventions addressing all 3 together could have multiplicative effects. eHealth interventions can be accessed privately, and evidence from general populations suggests these can effectively address all 3 health outcomes. However, it is unclear how useable, accessible, or acceptable eHealth interventions are for MSM and what factors affect this.
We undertook a systematic review of eHealth interventions addressing sexual risk, substance use, and common mental illnesses among MSM and synthesized evidence from process evaluations.
We searched 19 databases, 3 trials registers, OpenGrey, and Google, and supplemented this by reference checks and requests to experts. Eligible reports were those that discussed eHealth interventions offering ongoing support to MSM aiming to prevent sexual risk, substance use, anxiety or depression; and assessed how intervention delivery or receipt varied with characteristics of interventions, providers, participants, or context. Reviewers screened citations on titles, abstracts, and then full text. Reviewers assessed quality of eligible studies, and extracted data on intervention, study characteristics, and process evaluation findings. The analysis used thematic synthesis.
A total of 12 reports, addressing 10 studies of 8 interventions, were eligible for process synthesis. Most addressed sexual risk alone or with other outcomes. Studies were assessed as medium and high reliability (reflecting the trustworthiness of overall findings) but tended to lack depth and breadth in terms of the process issues explored. Intervention acceptability was enhanced by ease of use; privacy protection; use of diverse media; opportunities for self-reflection and to gain knowledge and skills; and content that was clear, interactive, tailored, reflective of MSM’s experiences, and affirming of sexual-minority identity. Technical issues and interventions that were too long detracted from acceptability. Some evidence suggested that acceptability varied by race or ethnicity and educational level; findings on variation by socioeconomic status were mixed. No studies explored how intervention delivery or receipt varied by provider characteristics.
Findings suggest that eHealth interventions targeting sexual risk, substance use, and mental health are acceptable for MSM across sociodemographic groups. We identified the factors shaping MSM’s receipt of such interventions, highlighting the importance of tailored content reflecting MSM’s experiences and of language affirming sexual-minority identities. Intervention developers can draw on these findings to increase the usability and acceptability of integrated eHealth interventions to address the syndemic of sexual risk, substance use, and mental ill health among MSM. Evaluators of these interventions can draw on our findings to plan evaluations that explore the factors shaping usability and acceptability.
Men who have sex with men (MSM) face disproportionate risks in relation to use of tobacco, alcohol, and legal and illegal drugs (henceforth termed
In addition to their effectiveness, it is important to examine what factors affect the usability and acceptability of eHealth interventions addressing these various outcomes among MSM. This should inform the development of interventions that can feasibly and acceptably address all 3 outcomes together [
Toward this end, we undertook a systematic review of eHealth interventions addressing these 3 outcomes and targeting this population. We included interventions addressing these outcomes together or separately, and aimed to synthesize evidence of effectiveness, describe intervention theories of change, and synthesize evidence from economic and process evaluations. This paper reports on the synthesis of process evaluations examining what factors related to interventions, providers, participants, or contexts (ie, environmental or structural factors) promote or impede delivery or receipt of these interventions.
Reports eligible for inclusion in the overall review reported on eHealth interventions that were delivered by mobile phone apps, the internet, or other electronic communication technology; offered ongoing support to populations consisting entirely or principally of gay, bisexual, and other men (including cisgender and transgender men) who have sex with men; and aimed to prevent HIV/sexually transmitted infections, sexual risk behavior, alcohol, tobacco or drug use, anxiety, or depression. We excluded interventions that offered one-off (rather than ongoing) support or that involved human providers (eg, in a chat room). Reports eligible for the process evaluation synthesis reported on characteristics of interventions, providers, participants, or context affecting delivery or receipt of eligible interventions. We included published and grey literature and set no restrictions by location or language.
Terms used in our search strategy covered 2 concepts joined by the Boolean operator “and”: MSM and eHealth. We searched 19 databases containing health and social science literature (from October 2018 to November 2018 and updated on April 2020). Our complete search strategy for the original OvidSP MEDLINE database is included in
Citations were uploaded to EndNote (Clarivate Analytics), deduplicated, and then uploaded to EPPI-Reviewer (version 4.0, EPPI-Centre) for screening. CB and JF independently screened titles and abstracts in batches of the same 50 references, resolving disagreements by discussion. After reaching an agreement rate of at least 95%, they single-screened all remaining citations. Screening of full texts followed a comparable process.
For process evaluations, CB and RM used an adapted version of an existing tool [
Using thematic synthesis methods [
First, CB and RM prepared tables describing the quality of each evaluation, intervention details, study site and population, and pertinent findings. Second, the reviewers independently piloted coding of 2 high-quality studies. Coding began with in vivo codes which closely reflected the words used in the findings sections. The reviewers then grouped and organized codes, applying axial codes that reflected higher-order, cross-cutting themes. They then met to compare and contrast their coding, developing an overall set of codes. Next, they each went on to independently code the remaining reports, drawing on the agreed set of codes and developing new in vivo and axial codes as new themes emerged. At the end of this process, they met to compare their sets of codes. They identified commonalities, differences of emphasis, and contradictions to develop an overall analysis which drew on the strengths of the 2 sets of codes and which resolved any contradictions or inconsistencies.
Our search retrieved 26,044 unique results, including 1 identified via reference checking (see
Searches and screening. EE: economic evaluation; OE: outcome evaluation; PE: process evaluation; ToC: theory of change.
Twelve reports were eligible for inclusion in the process evaluation synthesis (see
Nearly all process evaluations explored ways in which intervention characteristics affected receipt, although the included reports tended to lack breadth in the areas explored and in-depth exploration of the findings that they did report [
Across health domains, acceptability was enhanced when interventions were easy to use and free of technical problems. Few technical problems were reported. For example, from studies assessed as medium reliability, 10% or fewer Smartphone Self-Monitoring users reported technical difficulties [
From studies of medium and high reliability, accompanying materials outside of the electronic environment (such as printable materials [
From studies of medium reliability across health domains, it was apparent that intervention content which involved clear and comprehensive information facilitated acceptability. For example, Queer Sex Ed participants appreciated that this intervention provided comprehensive information on a range of sexual health and relationship topics rather than focusing narrowly on sexually transmitted infections [
Fun [
Language and tone emerged as an important aspect of acceptability across interventions addressing all 3 health domains and in studies of medium [
There were also some challenges in getting the language right for MSM-specific interventions. Some users of the Rainbow SPARX and Online Mindfulness-Based Cognitive Therapy interventions suggested that the sexuality-related terminology could be improved [
Participants in studies of medium [
In studies of medium reliability, privacy and intrusiveness emerged as important themes influencing acceptability across 2 interventions, which between them addressed all 3 health outcomes [
The pacing and structuring of content influenced acceptability across health domains. In studies of medium [
Users liked when intervention content progressed in a cumulative way [
Program length arose as a common theme affecting the acceptability of some modular interventions. Users of the 8-session Online Mindfulness-Based Cognitive Therapy [
Participants valued that interventions were designed for people like them. From studies of high reliability, it was apparent that participants valued when interventions presented realistic scenarios and examples and addressed issues relevant to their own lives [
Users of the Rainbow SPARX and Queer Sex Ed interventions liked that these programs were “[lesbian, gay, bisexual, and transgender] LGBT–specific” [
Online Mindfulness-Based Cognitive Therapy users had mixed views on how effectively this intervention was tailored for people like them [
In studies of medium [
Findings from the evaluation of the Smartphone Self-Monitoring intervention (targeting sexual health, substance use, and mental health outcomes) suggest that some participants valued its daily, mobile-based self-monitoring in contrast to the comparison group’s biweekly, web-based approach. One user described the benefits this way [
Helps me keep a “log”, like therapy—but can do it every day instead of waiting for a week to see your therapist…Nice to do it throughout the day, multiple times a day, on a daily basis. Life happens daily—not weekly like when you see a therapist.
Participants in 3 interventions, which between them addressed all 3 health domains, highlighted the opportunities for introspection and self-reflection that the interventions presented [
I started changing my behavior once I started taking the surveys—I have been thinking about it for a while but the surveys make me concentrate on certain areas of my life that I wasn’t focusing on.
A few also reported that engaging in self-monitoring across multiple domains enhanced their awareness of the relationships between their substance use, sexual behaviors, and other triggers for drug use [
I was able to see mistakes that I make in the actions of the characters. I wasn’t completely aware of my behavior until I judged a character’s behavior and then compared the same behavior to my own.
Participants in the Smartphone Self-Monitoring intervention, which addressed all 3 health outcomes, valued the opportunity for self-expression that the intervention offered, as described by this participant: “I feel free to vent to the phone about things that I can’t talk to my partner about—I can really express how I feel” [
Evaluations of 4 interventions (2 targeting sexual health alone [
A medium-reliability study of the HealthMindr mobile phone app found no differences in the time spent on the app by participant location in different cities in the United States, age, ethnicity, or knowledge of local HIV testing [
Madkins et al [
Exploring intervention receipt qualitatively, a high-reliability study found that for Rainbow SPARX, a computer game intervention for sexual minority youth aged 13-19 years, some older users reported that some aspects were too easy and the program “babied” them [
I thought some things were a little easy…Like overall it wasn’t difficult to figure out what you needed to do. Those little puzzles were quite easy to do. I guess it would be hard to make them more difficult though because you would have to be careful that everyone could actually get it.”
Qualitative research with participants of Rainbow SPARX and Smartphone Self-Monitoring found that these interventions could play a role in complementing the external mental health support participants were receiving [
Few studies explored how the context for using the intervention was associated with the experience of its use. Those that did focused on internet speed in the high-reliability 2007 [
One-third of reports included in the overall review included process evaluation data. All but one process evaluation took place in the United States. Most interventions targeted a single health domain of interest for this review (sexual health, substance use, or mental health), with the majority focused on sexual health. However, 2 aimed to address aspects of all 3 [
Process evaluations rarely explored how intervention receipt varied between contexts. We found no eligible reports examining what factors affected intervention delivery as opposed to receipt. This seems to reflect the emerging state of process evaluations in eHealth literature, with other reviews of eHealth interventions reporting a similar pattern [
In terms of intervention characteristics, as with use of eHealth interventions among general populations [
Participants liked content that was interactive and aesthetically pleasing, and they enjoyed the use of diverse media such as animations, videos, and graphics. However, among rural MSM these media could also reduce loading times for users with slower internet connectivity. Although modular approaches could support users to absorb program content cumulatively, interventions that were too long detracted from acceptability, with some users preferring that less or no time be required between sessions. The ideal number and length of modules is likely dependent on a variety of participant, intervention, and contextual factors.
Individual tailoring based on participant characteristics and risk profiles increased acceptability, highlighting this as a particularly promising approach and aligning with other studies of eHealth behavioral interventions [
As with studies of eHealth interventions for general populations [
Although reviews of eHealth interventions for general populations report higher use and engagement among participants with higher levels of education [
Our process evaluation synthesis was limited by the size and quality of eligible reports. Most were assessed as medium- or high-quality in terms of their reliability and usefulness. However, studies often lacked depth and breadth of analysis, and only around half were judged to privilege MSM’s perspectives.
The vast majority of interventions targeted MSM only and all were evaluated principally among MSM, although 3 were assessed among samples that included cisgender women [
eHealth interventions offer an avenue for MSM to access behavior change interventions privately, anonymously, and at times they find convenient. This synthesis identified several factors shaping MSM’s receipt of eHealth interventions addressing substance use, mental ill health, and sexual risk. Its findings suggest such interventions are acceptable for MSM across sociodemographic groups, although evidence in this area is limited and mixed. Different content for younger and older adolescents might be warranted. Variation in engagement and acceptability by participant characteristics should be explored in future research, and new interventions should be rigorously piloted to refine aspects affecting usability and acceptability [
Our review has identified several intervention characteristics affecting acceptability that existing research suggests are applicable to eHealth interventions for MSM and non-MSM populations alike. These include aspects of usability, length, aesthetics, multimedia use, and tailoring to participants’ personal and risk characteristics [
Our findings regarding the value that participants place on interventions that address the reality of their lives and the interrelationships between the different domains of health suggest that eHealth interventions simultaneously addressing sexual health, substance use, and mental health might be particularly acceptable. Our review of theories of change [
Search terms and strategy for the MEDLINE database.
Descriptions of interventions included in process synthesis.
Characteristics and quality of appraisal of process evaluations.
Coding structure for process evaluation synthesis.
lesbian, gay, bisexual, and transgender
men who have sex with men
National Institute for Health Research Public Health Research Programme
This work was funded by the National Institute for Health Research Public Health Research Programme (NIHR PHR; PHR 17/44/48). This report presents independent research commissioned by NIHR PHR. The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the National Health Service, the NIHR, the Medical Research Council, the Central Commissioning Facility, The NIHR Evaluation, Trials and Studies Coordinating Centre, the PHR Programme, or the Department of Health. The funder had no role in study design, in the collection, analysis or interpretation of data, or in writing this manuscript.
CB conceptualized and led the design of the study. AM, GJMT, JF, PW, RM, and TCM contributed to the development of the study’s methods. RM and CB developed the intervention typology and quality-assessed and synthesized process evaluation studies. RM led the drafting of the manuscript, with significant input from AM, CB, GJMT, JF, PW, and TCW.
None declared.