As smartphones have become an integral part of modern daily life [1,2], mobile health (mHealth) interventions for treatment adherence and self-management promotion have rapidly developed. These interventions often use existing smartphone features, such as sending SMS text notifications as cues to take prescribed medications [3]; others may integrate external technologies, such as a Bluetooth-enabled glucometer linked to a smartphone app to track blood glucose levels over time with the goal of supporting diabetes management [4]. In general, more frequent engagement, or interaction, with mHealth tools is expected to result in improved treatment adherence or self-management [5]. However, mHealth tools are frequently abandoned by users. Among mobile phone users in the United States, more than half reportedly downloaded an mHealth app but nearly half also stopped using the app due to high data entry burden, low interest, and costs [6].

There is a disconnect between research findings supporting positive correlations between engagement and adherence or self-management and user tendencies to stop using mHealth tools. This discrepancy may reflect an argument put forth by Arigo et al [7] that the mHealth field “lacks a science of engagement.” Specifically, there is (1) a lack of consensus in how mHealth user engagement is measured, defined, and reported; (2) no consensus on the optimal level of mHealth user engagement to facilitate meaningful behavior change; and (3) infrequent consideration of user engagement as a research end point. Not treating user engagement as a research end point suggests that this domain is poorly defined and haphazardly evaluated, particularly in terms of how user engagement might evolve over the course of the intervention or relate to behavioral and health outcomes. While the tendency for mHealth use to decline over time suggests that users will not experience maximum benefit from accessing these tools, poor measurement of user engagement has presented a challenge to researchers’ abilities to characterize exactly how and why engagement may decrease and how these decreases may affect intervention outcomes. These critical gaps in the science of user engagement significantly limit the utility, effectiveness, uptake, and scalability of mHealth interventions for adherence and self-management promotion.

Recent systematic reviews have examined aspects of user engagement with mHealth interventions for specific diagnoses, including hypertension [8], physical activity [9,10], depressive symptoms [11,12], and mental health conditions [13]. The applicability of these reviews to adherence and self-management mHealth interventions is significantly limited by the small number of studies included and a lack of unified focus on adherence and self-management behaviors. These prior reviews have been further limited by using a very broad definition of engagement to include usability, feasibility, user satisfaction, and acceptability [13], limiting the review to studies in which only postassessment retention data were obtained [11] (excludes interventions earlier in the design phase), and focusing on design features associated with user engagement rather than the evaluation of user engagement itself [10]. Another review identified a range of valid and reliable measurement approaches for evaluating user engagement with mHealth interventions for behavior change but used a snowballing method to identify sources rather than a rigorous systematic review of the literature [14]. To encourage the continued use of, and optimal engagement with, mHealth interventions to facilitate adherence and self-management behavior change, it is imperative to comprehensively and systematically evaluate the recent scientific landscape of mHealth user engagement with a clear focus on adherence and self-management behavior.

In response to gaps identified in the current scientific literature [7], our registered systematic review aimed to (1) characterize user engagement with interventions promoting mHealth treatment adherence or self-management for adults and youth with health conditions and (2) generate user engagement–focused research recommendations. “User engagement” with mHealth tools can be conceptualized as both behavioral (the extent of use) and experiential (the subjective experience of interacting with the technology) [15]. To enhance the practical application of our review findings, we focused on the behavioral aspects of user engagement to evaluate the degree of use and interaction with the mHealth tool [7] among users with chronic health conditions. Users’ behavioral interaction with mHealth tools, features, and associated behavior change components is known as “Little e” engagement. In theory, increased “Little e” engagement is expected to contribute to increased engagement in the desired health behavior, known as “Big E” [16]. Thus, better precision in how behavioral interaction, or “Little e” engagement, is empirically evaluated could help to facilitate greater changes in “Big E” outcomes, thus improving the overall efficacy of mHealth interventions for adherence and self-management. A systematic review approach was selected due to expected heterogeneity in both the measurement of user engagement and adherence and self-management outcomes, which precludes the use of a meta-analysis [17-19]. We specifically aimed to answer the following research questions: (1) How is user engagement defined and measured in studies of mHealth interventions to promote adherence to prescribed medical or health regimens or self-management among people living with a health condition? (2) To what degree are patients engaging with these mHealth interventions? (3) What is the association between user engagement with mHealth interventions and adherence or self-management outcomes? (4) How often is user engagement a research end point?

We also developed the following exploratory question: are there differences in user engagement measurement approaches and levels between studies that provide monetary compensation and those that do not?

Institutional review board approval was not required because the study was not considered human participant research.

The search strategy (Multimedia Appendix 1) was developed by the first and last authors (CE and KR) in collaboration with an informationist at Johns Hopkins Libraries; implemented in Ovid MEDLINE, Embase, Web of Science, PsycINFO, and CINAHL; and restricted to manuscripts published between 2016 and 2021. All citations returned from the search were imported into our Covidence (Veritas Health Innovation Ltd) [20] database for screening and data extraction.

The inclusion and exclusion criteria are presented in Textbox 1.

Citations were imported into Covidence; duplicate citations were removed; and title and abstract screening was conducted, followed by a full-text review. At each level of review, 2 separate review team members evaluated each article against the inclusion and exclusion criteria, with discrepancies resolved by CE and N Muthusamy. Any study meeting the inclusion criteria after the full-text review progressed to the data extraction phase (detailed in the next subsection).

We decided not to conduct a formal risk-of-bias assessment, given that the primary aim of this review was to characterize the evaluation and measurement of user engagement rather than certainty in the primary study results. Therefore, it was deemed inappropriate to evaluate the studies using standard risk-of-bias assessment tools.

Statistical analyses were conducted using SPSS software (version 28.0; IBM Corp) [21]. The extracted data were summarized using frequencies and percentages. Methods of measuring user engagement with the mHealth intervention were thematically grouped into discrete measurement categories. Subgroup sensitivity analyses were conducted to examine studies involving pediatric samples (participants aged 0-18 y or aged up to 25 y if the sample was characterized as “pediatric” by the authors) separately from those involving adults only (participants aged >18 y). In these age-based subgroup analyses, of the 292 included studies, 5 (1.7%) were excluded due to including both pediatric and adult participants, and 1 (0.3%) was excluded due to not reporting participant ages. Given that this investigation was designed as a systematic review, no effect measures or meta-regressions were used. No missing summary statistics or data conversions were used.

Figure 1 presents the PRISMA flow diagram of the screening process. The initial search returned 3955 citations, from which 70 (1.77%) duplicates were removed. During title and abstract screening, the remaining 3885 studies were screened, and 2736 (70.42%) were excluded. During full-text screening, the remaining 1149 studies were evaluated, and 857 (74.59%) were excluded. The primary reasons for exclusion were as follows: ineligible manuscript type (eg, published abstract; 330/857, 38.5%), user engagement with the mHealth intervention was not measured (206/857, 24%), and ineligible participant population (111/857, 12.9%). The final review included 292 studies [22-313] (refer to Multimedia Appendix 2 for all included studies and characteristics).

Nearly half of the studies (135/292, 46.2%) were conducted in the United States and used a randomized controlled trial design. The median number of participants was 77 (IQR 34-164). Nearly half (135/292, 46.2%) were considered feasibility studies. The median study length was 90 (IQR 60-180) days. Diabetes (51/292, 17.5%) and mental health conditions (35/292, 11.9%) were the most common diagnoses. Study characteristics were similar between adult and pediatric studies, with the exception of health conditions, reflecting expected age-based differences in diagnoses more common among adults than among children (eg, type 2 diabetes and substance use were more commonly studied in adult samples than in pediatric samples).

Table S1 in Multimedia Appendix 3 contains specific details on the mHealth interventions’ adherence or self-management targets, intervention components, and intended users. The most frequently targeted adherence or self-management concern was taking medication (98/292, 33.6%), followed by exercise (93/292, 31.8%) and diet (73/292, 25%). Nearly all mHealth interventions were used by the patient (291/292, 99.7%), but some of them included health care providers (58/292, 19.9%) or caregivers (20/292, 6.8%). The majority of mHealth interventions comprised a mobile app (220/292, 75.3%), SMS text messaging or push notifications (74/292, 25.3%), websites or web portals not within a mobile app (48/292, 16.4%), and nonwearable monitoring devices (47/292, 16.1%). Interventions that did not include a mobile app primarily comprised text messaging, a wearable device, a nonwearable device, video web conferencing or telephone calls, or a website or web portal not within a mobile app. Nearly three-quarters of the interventions (216/292, 74%) prompted users to engage with the mHealth intervention. Less than half of the studies (124/292, 42.5%) provided monetary compensation for participation. Only 10.6% (31/292) allowed users to continue using the mHealth intervention after the formal study period. Intervention characteristics were generally similar between adult and pediatric studies, with the exception of intervention target behavior, reflecting expected age-based differences in health concerns more common among adults than among children (eg, exercise, mental health management, and drug or alcohol use or abuse were more commonly targeted in adult samples than in pediatric samples).

The association between engagement and treatment adherence or self-management outcomes was only assessed in a little more than one-third of the studies (103/292, 35.3%). Among these 103 studies, 60 (58.3%) tended to report and frame results and conclusions to suggest that higher engagement was associated with positive adherence or self-management outcomes. These results were similar between adult and pediatric studies (Table 3).

User engagement was a research end point in only 19.2% (56/292) of the reviewed studies, with similar results in adult (44/241, 18.3%) and pediatric (11/45, 24%) studies. Of these 56 studies, 30 (54%) used nonrandomized experimental designs, and 27 (48%) were feasibility studies. User engagement was typically defined as “engagement” (24/56, 43%), “adherence” (16/56, 29%), or “use” (16/56, 29%; Table S2 in Multimedia Appendix 3) and was most frequently measured with user log-in data (31/56, 55%) or manual data entry (22/56, 39%; Table S5 in Multimedia Appendix 3).

Whether study participants were provided monetary compensation for their participation or not (or if compensation was not reported), user engagement measurement methods were used in similar proportions (<10% difference). Similarly, user engagement levels were observed in similar proportions between studies that provided monetary compensation and those that did not (or did not report compensation; Table S6 in Multimedia Appendix 3).

Of the 292 studies, 10 (3.4%) included as an intervention component financial incentives for using the technology or meeting mHealth intervention goals. Of these 10 studies, 7 (70%) reported user engagement levels. Among these 7 studies, in 1 (14%), the user engagement level was characterized as “high”; in 1 (14%) as “medium”; and in 1 (14%), as “low”; and 4 (57%) reported >1 level due to multiple measurement or analytic approaches.

The principal findings relative to our specific research questions are provided in the following subsections.

Terms used to describe user engagement outcomes were wide ranging, but the most commonly used were “use” (102/292, 34.9%) and “engagement” (94/292, 32.2%). Across all studies reviewed, 11 distinct user engagement measurement approaches were identified, comprising both objective and subjective methods. The most common methods were user log-in data from the app or web portal (160/292, 54.8%), manually entering data in an app (77/292, 26.4%), qualitative interviews (54/292, 18.4%), and responding to SMS text messages or push notifications (49/292, 16.8%).

User engagement level was difficult to quantify because it was only characterized in two-thirds of the reviewed studies (195/292, 66.8%), of which a little more than half (99/195, 50.8%) reported >1 level of engagement due to the use of multiple measurement methods or analyses. Only one-third of the studies (99/292, 33.9%) evaluated change in engagement over time, which tended to decrease.

Only one-third of the studies (103/292, 35.3%) evaluated the association between engagement and treatment adherence or self-management outcomes. When evaluated, the study authors tended to report and frame results and conclusions to suggest that higher engagement was associated with positive adherence or self-management outcomes.

User engagement was rarely considered a research end point (56/292, 19.2%).

No; whether study participants were provided monetary compensation for their participation or not (or if compensation was not reported), user engagement methods were used in similar proportions, and user engagement levels were observed in similar proportions.

Despite immense focus in both commercial and research sectors on using mHealth to support chronic illness treatment adherence and self-management [314-316], people do not remain engaged with mHealth interventions in the long term [6]. Our systematic review also found that user engagement tends to decline over time. Consistent with the “Little e, Big E framework” [16], mHealth intervention success hinges on the expectation that people will interact with the technology and thereby experience intended behavior changes that will lead to better overall health. Most importantly, our systematic review revealed critical limitations with how user engagement is defined and evaluated, which significantly impedes our ability to (1) communicate about this topic and (2) draw strong conclusions about how much user engagement is necessary to achieve desired behavior and health changes.

A principal finding of our review was that there is no agreed-upon definition of mHealth user engagement, which is a direct barrier to interdisciplinary communication about this topic. We found >60 terms used to define user engagement, even when limiting inclusion criteria to the behavioral evaluation of this concept [15]. While “use” (102/292, 34.9%) and “engagement” (94/292, 32.2%) were the most common terms, “adherence” (59/292, 20.2%) also appeared frequently. “Adherence” as a term for mHealth user engagement can be problematic because it is closely tied to treatment adherence [317], a key target of many of the mHealth interventions included in this systematic review. “Adherence” can apply to engaging with the mHealth technology as intended, such as logging in to an app [109] or responding to SMS text messages [243] rather than following a prescribed treatment regimen. Engaging with the technology can have a direct connection to treatment adherence (eg, using a Bluetooth-enabled blood pressure cuff reflects adherence to a key part of the hypertension management regimen as well as engaging with the mHealth technology [118,308]). However, such interchangeability and a lack of consistency in terminology will continue to be a barrier to communicating within and outside the field and add to existing challenges with defining and evaluating this domain. The extremely wide range of terms used to define user engagement likely reflects a major critique of mHealth research—that there is a lack of a science of engagement [7]. We encourage standardization in terminology used when the intention is to evaluate users’ behavioral engagement with mHealth technology, rather than treatment regimen, with the most common terms found in this systematic review: “use” or “engagement.” Integrating standard user engagement language in an internationally adopted clinical terminology system, such as the Systematized Nomenclature of Medicine–Clinical Terms, could help facilitate standardization efforts.

Another key finding was that the use of user log-in data was the default mHealth user engagement outcome, except for wearing a device when the intervention involved a wearable device component and responding to SMS text messages and push notifications when the intervention involved SMS text messaging and push notifications. Reliance on user log-in data over the other 10 major measurement approaches likely reflects that these data can be relatively straightforward to extract from a web portal. A strength of the use of user log-in data is that it may indicate more effortful and deliberate interactions with the technology (the user likely needs sufficient motivation to open and navigate an app) compared with, for example, quickly responding to a SMS text message or wearing a pedometer on the wrist. User log-in data may be a more accurate reflection of engagement behavior than self-report, which is potentially subject to social desirability and recall bias. The tendency for engagement assessed using user log-in data to be characterized as “low” may reflect the higher user burden of logging in to, and interacting with an, app, in addition to the overwhelming selection of this method for measuring user engagement. Reliance on user log-in data may bias interpretations of user engagement to favor higher quantities of engagement at the expense of higher-quality engagement (eg, infrequent access of a particularly effective app feature that users perceive as interesting, helpful, or motivating may be higher-quality engagement compared with frequent log-ins to the app homepage). Given the ubiquitous selection of user log-in data and the heterogeneity of adherence and self-management outcomes in this systematic review, it is not possible to evaluate which user engagement metric is best. In general, investigators should select the user engagement metric reflecting essential interactions with the mHealth intervention’s key technology needed to facilitate behavior and health changes.

We found that the exact degree of user engagement with the interventions reviewed in this study was difficult to estimate and that the association between user engagement and adherence or self-management outcomes were rarely evaluated. Both findings may relate to an important secondary finding that researchers rarely set a priori benchmarks for how much user engagement is considered scientifically adequate (no minimum engagement research benchmark set). Concerningly, when no minimum research benchmark was set, researchers tended to characterize user engagement as “high,” suggesting overoptimism about study results in the absence of any hypothesized lower boundary for what constituted minimally acceptable mHealth use. Setting cut points for how much user engagement is hypothesized to be adequate would also help to guide how much and in what ways users should interact with the mHealth intervention components to experience meaningful improvements in target outcomes. Matching technology dosages to minimum engagement research benchmarks, when possible, could inform whether engagement recommendations were met and allow researchers to evaluate whether exceeding expectations is associated with even greater improvement in behavior or health outcomes. Of note, studies with technology dosages or minimum engagement research benchmarks seemed to demand more from participants, as evidenced by the more common use of user log-in data and manual data entry to evaluate user engagement. User engagement burden should be considered when setting mHealth engagement expectations.

Setting minimum engagement research benchmarks could also account for variability in user engagement–level characterization depending on the metric selected and help to define what constitutes effective and meaningful engagement. Specifically, engagement assessed using user log-in data tended to be characterized as “low.” By contrast, responding to text notifications or interacting via chats, phone calls, or social media posts, which may be more socially rewarding and less effortful to complete, tended to be characterized as “high.” A similarly higher pattern of user engagement was observed in studies evaluating the use of wearable (6/63, 10% vs 0) and nonwearable devices (7/63, 11% vs 0), which is often part of the users’ regular self-management routines (eg, Bluetooth-enabled pill bottle or glucometer) or passively worn (eg, pedometer) and thus may be less burdensome to use. A higher SMS text message response rate and comparatively lower app log-in rate may both be minimally acceptable and potentially meaningful levels of engagement to effectively facilitate behavior or health change, particularly considering the level of effort and motivation involved in each type of interaction. Given the inconsistencies in how investigators characterize engagement levels, setting minimum engagement research benchmarks could inform whether user engagement is adequate across metrics and associated with key adherence and self-management outcomes. This approach could also improve precision in how engagement levels are characterized (ie, what level of engagement is considered “low” vs “high”?). Such efforts during the study planning phase could reduce the tendency to characterize user engagement at multiple levels due to the use of multiple metrics and analyses. Rethinking engagement levels as “adequate,” “acceptable,” “effective,” or “meaningful” versus “high” or “low” may better inform how much people should realistically use mHealth technology and evaluate whether engagement level is associated with behavior and health changes.

In the mHealth trials reviewed, an important secondary finding was that participants were rarely told how much they should aim to engage with the technology (no technology dosages). This is in contrast to drug trials in which participants are instructed to take a specific dosage of medication on a strict schedule and trials of in-person behavior change interventions (traditional therapy) in which participants are given therapy regimens and expectations for participation (eg, attend 12 therapy sessions and complete assigned homework each week of treatment) [11]. Although an early-phase research goal may be to see how much people interact with the technology in the absence of recommendations, an important future goal is to understand how much engagement is needed to maximize behavior and health changes. To this end, recommending in what ways and how much users should aim to engage with the technology is necessary. Giving technology dosages would also allow for experimental evaluation of different levels of recommended technology use to see what recommendation helps users achieve the most positive behavior and health changes. Furthermore, improvements in health and wellness can take time, and mHealth users may not observe immediate benefits to using mHealth interventions, which could contribute to premature abandonment of the tool. Providing users with clearer guidance on how much and what type of mHealth intervention use is needed to begin effecting positive behavior and health changes could help encourage sustained use.

Another primary finding was that user engagement was rarely considered a research end point. This is a problem because it is generally expected that users will need to interact with the mHealth technology to experience clinical benefit. Thus, user engagement behavior is a critical aspect of the mHealth intervention itself. Without designating user engagement as a research end point, mHealth intervention trials risk lacking the necessary empirical data and results to help end users understand how to optimally use the technology to see maximal clinical benefit. Lacking these data and results may also be a barrier to informing how these digital tools can be incorporated into regular clinical practice and policy [22]. Thus, in addition to carefully measuring the treatment adherence, self-management behavior, and associated health outcomes directly targeted by the mHealth intervention, equal care should be given to evaluating user engagement, such as how user engagement with the technology may change over time and potentially influence outcomes. Shifting user engagement from afterthought data to an actual research end point would likely improve the quality and rigor of mHealth research and help develop mHealth interventions that motivate users to interact with the technology [318], and, ideally, experience greater clinical benefit.

Our exploratory analysis of studies that provided monetary compensation for study participation versus those that did not (or did not report monetary compensation) showed that engagement-level characterizations were similar between the groups. It is possible that the monetary compensation was provided primarily for completing study procedures (eg, completing study surveys) rather than engaging with the mHealth intervention. In the studies of interventions comprising financial incentives for using the technology or meeting mHealth intervention goals (10/292, 3.4%), no clear pattern emerged for user engagement levels, although this likely reflects the small number of studies incorporating this mHealth component. User engagement evaluation methods were used in similar proportions between the groups, suggesting that monetary compensation was not associated with user engagement measurement selection. An important avenue for future mHealth engagement research is to improve the understanding of how monetary compensation is associated with user engagement levels, particularly when compensation is directed toward mHealth intervention use.

This systematic review is among the most comprehensive reviews on this topic, but it has some limitations. First, due to heterogeneity in the study outcomes, we did not conduct a meta-analysis. Thus, this systematic review cannot conclude which user engagement outcome is associated with the most effective interventions. Second, although we found that study authors tended to frame their results and conclusions to suggest that higher user engagement was associated with positive intervention outcomes, this finding should be considered preliminary and hypothesis generating and does not prove that higher engagement leads to better adherence or self-management, particularly given that study authors may have a tendency to report favorable results and frame their findings in a positive light. However, this is an important area to examine in future research to better understand the characteristics of users with higher mHealth engagement as it relates to adherence and self-management outcomes (eg, highly engaged users may represent a patient group with already high adherence and self-management). Third, we did not exclude studies on the basis of their attrition rate; a future direction for improving the science of engagement is to consider how dropout and withdrawal may influence the characterization of user engagement levels. Fourth, a traditional risk-of-bias assessment [319,320] was not conducted due to our interest in user engagement rather than the primary study results targeting adherence and self-management. If user engagement becomes more commonly designated as a research end point, such evaluations of study quality specific to the evaluation of user engagement will be warranted. Fifth, we included a range of medical and mental health conditions but, due to small sample sizes, were unable to conduct subgroup analyses by diagnosis; as the literature on user engagement grows, it may be possible to explore such differences by diagnosis. Sixth, we focused on the behavioral evaluation of user engagement, although this domain likely involves more than behavior alone [15,321], and these other aspects of user engagement may be examined in a future systematic review. Seventh, our interest was in the measurement and evaluation of mHealth user engagement; thus, we did not consider the numerous individual-level factors that could be related to user engagement. Eighth, although study location was not part of the inclusion criteria, nearly half of the reviewed studies (135/292, 46.2%) were conducted in the United States, which may have influenced the types of intervention studies and approaches to evaluating user engagement. Ninth, we conducted our search in specific scientific databases and did not review gray literature or unpublished results; although our search resulted in 292 studies meeting our inclusion criteria, this systematic review may not be generalizable to all mHealth interventions. Finally, it is necessary to select a cutoff date when conducting a systematic review; yet, new studies are constantly being published. An updated review of the mHealth user engagement literature may be warranted in the future. As the science of mHealth user engagement improves, an important next step is to evaluate how factors related to diversity, equity, and inclusion relate to user engagement to promote wider mHealth use and access.

Our systematic review on mHealth user engagement highlighted critical gaps in mHealth adherence and self-management literature as well as opportunities to improve research in this important area of digital health. mHealth researchers need to prioritize the evaluation of user engagement during the study planning phases. Strengthening mHealth user engagement methodological rigor would likely lead to higher-quality data and more impactful study results to guide practice and policy and ultimately encourage the uptake of interventions promoting mHealth adherence or self-management. Furthermore, improved user engagement evaluation may help researchers to identify effective strategies for supporting meaningful user engagement and sustained interest to help users fully experience the intended behavioral and health benefits of the mHealth intervention [16]. Such research is critical for building the evidence base needed to integrate mHealth interventions into regular clinical care and practice guidelines.

The following recommendations for studies of mHealth user engagement are offered: