JMIR J Med Internet Res Journal of Medical Internet Research 1438-8871 JMIR Publications Toronto, Canada v23i2e23180 33605897 10.2196/23180 Review Review Associations Between Digital Health Intervention Engagement, Physical Activity, and Sedentary Behavior: Systematic Review and Meta-analysis Eysenbach Gunther O'Brien Heather Vandelanotte Corneel Mclaughlin Matthew BSc 1
School of Medicine and Public Health University of Newcastle University Drive Callaghan, 2308 Australia 61 02 4924 6477 Matthew.Mclaughlin1@health.nsw.gov.au
2 3 4 https://orcid.org/0000-0003-2870-8556 Delaney Tessa BND 1 2 3 4 https://orcid.org/0000-0002-2756-4821 Hall Alix PhD 1 2 3 4 https://orcid.org/0000-0002-1043-6110 Byaruhanga Judith MPH 1 2 3 4 https://orcid.org/0000-0003-0382-1552 Mackie Paul MSc 5 6 https://orcid.org/0000-0001-5840-4348 Grady Alice PhD 1 2 3 4 https://orcid.org/0000-0003-3379-5340 Reilly Kathryn PhD 1 2 3 4 https://orcid.org/0000-0002-2862-956X Campbell Elizabeth PhD 1 2 3 4 https://orcid.org/0000-0002-7265-5407 Sutherland Rachel PhD 1 2 3 4 https://orcid.org/0000-0003-0470-7663 Wiggers John PhD 1 2 3 4 https://orcid.org/0000-0001-6361-9685 Wolfenden Luke PhD 1 2 3 4 https://orcid.org/0000-0002-6178-3868 School of Medicine and Public Health University of Newcastle Callaghan Australia Hunter New England Population Health Wallsend Australia Hunter Medical Research Institute New Lambton Heights Australia Priority Research Centre for Heath Behaviour University of Newcastle Callaghan Australia School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle Callaghan Australia Centre for Research Excellence in Stroke Recovery and Rehabilitation, Florey Institute of Neuroscience Melbourne Australia Corresponding Author: Matthew Mclaughlin Matthew.Mclaughlin1@health.nsw.gov.au 2 2021 19 2 2021 23 2 e23180 4 8 2020 11 9 2020 18 11 2020 7 12 2020 ©Matthew Mclaughlin, Tessa Delaney, Alix Hall, Judith Byaruhanga, Paul Mackie, Alice Grady, Kathryn Reilly, Elizabeth Campbell, Rachel Sutherland, John Wiggers, Luke Wolfenden. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 19.02.2021. 2021

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

 Background

The effectiveness of digital health interventions is commonly assumed to be related to the level of user engagement with the digital health intervention, including measures of both digital health intervention use and users’ subjective experience. However, little is known about the relationships between the measures of digital health intervention engagement and physical activity or sedentary behavior.

 Objective

This study aims to describe the direction and strength of the association between engagement with digital health interventions and physical activity or sedentary behavior in adults and explore whether the direction of association of digital health intervention engagement with physical activity or sedentary behavior varies with the type of engagement with the digital health intervention (ie, subjective experience, activities completed, time, and logins).

 Methods

Four databases were searched from inception to December 2019. Grey literature and reference lists of key systematic reviews and journals were also searched. Studies were eligible for inclusion if they examined a quantitative association between a measure of engagement with a digital health intervention targeting physical activity and a measure of physical activity or sedentary behavior in adults (aged ≥18 years). Studies that purposely sampled or recruited individuals on the basis of pre-existing health-related conditions were excluded. In addition, studies were excluded if the individual engaging with the digital health intervention was not the target of the physical activity intervention, the study had a non–digital health intervention component, or the digital health interventions targeted multiple health behaviors. A random effects meta-analysis and direction of association vote counting (for studies not included in meta-analysis) were used to address objective 1. Objective 2 used vote counting on the direction of the association.

 Results

Overall, 10,653 unique citations were identified and 375 full texts were reviewed. Of these, 19 studies (26 associations) were included in the review, with no studies reporting a measure of sedentary behavior. A meta-analysis of 11 studies indicated a small statistically significant positive association between digital health engagement (based on all usage measures) and physical activity (0.08, 95% CI 0.01-0.14, SD 0.11). Heterogeneity was high, with 77% of the variation in the point estimates explained by the between-study heterogeneity. Vote counting indicated that the relationship between physical activity and digital health intervention engagement was consistently positive for three measures: subjective experience measures (2 of 3 associations), activities completed (5 of 8 associations), and logins (6 of 10 associations). However, the direction of associations between physical activity and time-based measures of usage (time spent using the intervention) were mixed (2 of 5 associations supported the hypothesis, 2 were inconclusive, and 1 rejected the hypothesis).

 Conclusions

The findings indicate a weak but consistent positive association between engagement with a physical activity digital health intervention and physical activity outcomes. No studies have targeted sedentary behavior outcomes. The findings were consistent across most constructs of engagement; however, the associations were weak.

 engagement adherence digital health intervention digital behavior change intervention physical activity sedentary behavior mobile phone Introduction

Physical activity of any intensity reduces the risk of death and noncommunicable diseases [1]. Sedentary behavior is highly prevalent, displaces time to be physically active, and is associated with noncommunicable diseases and premature death [2,3]. As such, efforts to increase physical activity and concurrently decrease sedentary behavior have been identified internationally as public health priorities [4].

Digital health interventions (DHIs) have the potential to address physical inactivity, as they are accessible by large proportions of the population and can be delivered with high effectiveness at a low cost [5,6]. The World Health Organization defines digital health as the use of digital, mobile, and wireless technologies to support the achievement of health objectives and is inclusive of both mobile health (mHealth) and eHealth [7], including mobile phones, portable computer tablets (eg, iPads), web-based interventions, smartphone apps, and wearable devices [8]. An attractive feature of DHIs is their capacity to be scaled for large populations while concurrently being tailored to specific target groups [7,9]. There has been a considerable investment in the development and research on DHIs to improve physical activity, with a rise of 26% per year in journal article publications since 2000 [10]. Furthermore, the use of DHIs to promote and support participation in physical activity has been recommended in the Global Action Plan on Physical Activity 2018-2030 [4,11].

Despite the significant growth in DHIs, there is a limited understanding of the extent to which DHIs impact physical activity outcomes. Overall, systematic reviews indicate that DHIs targeting adult populations may have a modest effect in improving physical activity when delivered web based [12-16] and through smartphone applications [12,13,17-19]. A recent meta-analysis found that per week, web-based interventions increased moderate-to-vigorous intensity physical activity by 13.4 minutes (95% CI 12.96-13.89) and steps by 2185 (95% CI 1765-2605) [15]. Similarly, another meta-analysis of web-based interventions targeting physical activity found that physical activity significantly improved in the short term (Cohen d=0.14). The review suggested that the small effect size may be attributed to a lack of engagement with web-based interventions [14]. Other meta-analyses of mobile phone app interventions have reported effects that favored the intervention but were not statistically significant [17,19], with the suggestion that a lack of engagement may explain the lack of evidence to support effectiveness.

Participant exposure needs to be sufficient for any DHI to have an effect [20]. Engagement has been defined as the (1) extent of DHI usage such as the frequency, duration, amount, and depth of the accessed DHI and (2) subjective experience characterized by attention, interest, and affect [20]. To our knowledge, only one systematic review has explored the association between objective levels of engagement with DHIs (usage measures) and physical activity or sedentary behavior [8]. The review by Donkin et al [8] explored the association between the level of engagement with DHIs (web-based interventions) targeting adults and a range of health outcomes [8]. Included studies predominately reported the measures of psychological health, dietary intake, weight management, and smoking. The results were reported narratively because of the diverse measures of engagement and health outcomes. Only 3 of the 33 studies included measures of physical activity (n=5 associations) [21-24], with no studies reporting sedentary behavior outcomes. The engagement measures explored by the 3 studies included in the Donkin review focused on logins (n=3), activities completed (n=1), and website exposure (n=1). Of the 3 physical activity studies, Marcus et al [22] showed that a higher number of logins were correlated with an increase in the physical activity from baseline to 12 months. Similarly, McKay et al [21] found that those who logged into the program on 3 or more occasions had greater increases in physical activity than those with fewer logins. McKay et al [21] additionally found website exposure (usage) to be associated with higher increases in physical activity. In contrast, Carr et al [23,24] showed that neither the number of logins nor the number of activities completed were associated with physical activity at 8 months.

Understanding the relationship between engagement and health outcomes is important because it provides an opportunity to optimize the impact of interventions [20,25]. Engagement is hypothesized to influence the relationship between a DHI and the mechanisms of action of the DHI (eg, skills, attitudes, beliefs, knowledge), which then leads to the target behavior (eg, physical activity) [20]. Even smaller improvements in the effectiveness of DHIs are important, given the potential reach of these interventions [6,10]. The previous review by Donkin et al [8] used a definition of engagement that focused on the usage and user-directed web-based interventions, which excluded smartphone apps and group-based DHIs [8]. Since 2010, when their search was conducted, there has been a large increase in the mobile- and app-based research applied to physical activity; hence, there is an increased opportunity to garner further understanding of this relationship [8,10]. A broader definition of engagement that encompasses subjective experience has also been developed [20]. To our knowledge, no review has explored the relationship between subjective experience with DHIs and physical activity or sedentary behavior [20]. Therefore, a more contemporary review of the evidence is warranted.

Objective

In this context, we aim to (1) describe the direction and strength of the association between engagement with DHIs and physical activity and/or sedentary behavior in adults and (2) explore whether the direction of association between DHI engagement and physical activity or sedentary behavior varies by the type of engagement (ie, subjective experience, activities completed, time, and logins).

 Methods Design

This review was prospectively registered with the International Prospective Register for Systematic Reviews (PROSPERO; CRD42018110657) and is reported in accordance with the Joanna Briggs Institute guidance for conducting systematic reviews of association [26].

 Search Strategy

Searches for peer-reviewed literature were undertaken with the assistance of a research librarian in 4 electronic databases: Embase, MEDLINE, PsycINFO, and Scopus (Multimedia Appendix 1). We searched for records from the database inception to December 2019. Searches were restricted to English. This review was conducted alongside another review that aimed to describe the association between DHI engagement and dietary intake (PROSPERO CRD42018112189 [27]). Therefore, dietary intake search terms were also included in the search strategy. We used the modified versions of published search filters for physical activity and sedentary behavior [28], engagement [20], and DHIs [20,29,30].

Additional Search Methods

We conducted hand searches of all the publications from January 2016 to December 2019 in the following journals: Journal of Medical Internet Research, JMIR mHealth and uHealth, JMIR Medical Informatics, and JMIR Public Health and Surveillance. We conducted gray literature searches in “Google.com/ncr” search engine and used the search terms Physical Activity or Sedentary Behavior and Engagement and Digital Health Intervention and screened the first 200 hits for relevance. We screened the reference lists of key systematic reviews of DHI engagement [8,20]. We also contacted the authors of included studies for other potentially relevant studies.

 Inclusion Criteria Types of Studies

We included study designs that examined a quantitative association between a measure of engagement with a DHI and a measure of physical activity and/or sedentary behavior. DHIs were defined as the use of digital, mobile, and wireless technologies to support the achievement of health objectives [7], inclusive of both mHealth and eHealth. We adopted Perski et al’s [20] definition of engagement, defined as both the extent of the usage of the DHI (amount, frequency, duration, and depth; eg, activities completed, time, and logins) and the subjective experience (characterized by attention, interest, and affect). DHIs included but were not limited to mobile phones, portable computer tablets (eg, iPads), web-based interventions, and smartphone apps. We included DHIs involving synchronous communication as part of the program (eg, web-based chat, teleconferencing). We applied no restrictions on the length of the follow-up period or the country of origin of the studies. We included studies that recruited participants in the real-world settings (ie, ecological studies) as well as nonecological studies (ie, those conducted under controlled research conditions, where repeated contacts with research staff, comprehensive assessments, and recruitment to the study occurs before the individual accessing the DHI) [31]. All the quantitative study designs were also included.

 Population

We included any study undertaken with adult users (aged ≥18 years) of a DHI targeting physical activity or sedentary behavior. The studies of participants who had access to a DHI and the opportunity to engage with the DHI were eligible.

 Exposure

We included studies reporting any measure of engagement with a DHI, defined as the extent of usage (eg, activities completed, time, and logins) or the subjective experience of users (eg, measures of attention, interest, and affect, including but not limited to enjoyment, satisfaction, user experience, and usability) [20]. Engagement can be collected by the DHI (eg, analytics), observation, surveys of DHI users, or other quantitative methods. We excluded the qualitative measures of engagement (eg, focus groups).

 Outcome

We included studies reporting any measure of physical activity or sedentary behavior, including but not limited to self-report (eg, minutes of moderate-to-vigorous physical activity, minutes of walking, self-reported steps, distance traveled) and measured by a device (eg, steps from pedometer, mobile phone data, accelerometers). These could be reported in specific settings (eg, while at work), periods of the day (eg, mornings), or as the whole day. We included both cross-sectional measures of physical activity (ie, one time point) and those studies with multiple time points calculating changes in the physical activity over time (ie, cohort studies).

 Exclusion Criteria

We excluded the following studies:

Case studies, letters to the editor, and qualitative studies.

Studies that targeted children and adolescents (<18 years).

Studies that purposely sampled or recruited individuals based on pre-existing health-related conditions, including chronic health conditions such as chronic pain, a diagnosis of chronic disease, communicable disease, or mental illness, given our interest in generalizing the findings to general community samples.

Studies in which the individuals engaging in the DHI were not the target of the physical activity or sedentary behavior intervention (eg, doctors engaging with a physical activity app for their patients).

Studies that included a non-DHI component within an intervention (eg, a face-to-face component and digital components). This step was taken to ensure that the measures of engagement reflected only the digital component and not the intervention more generally.

Interventions not functioning at computer- or internet-based capacity (eg, SMS, CD-Rom, and computer-based interventions) to focus on more contemporary DHIs.

Those that targeted multiple health behaviors for the prevention of chronic disease (eg, sleep and physical activity or diet and physical activity) to reduce heterogeneity between health behaviors.

Studies where the full text was not available.

 Data Collection and Analyses Selection of Studies

After removing the duplicates, the authors (MM, TD, and JB) single-screened titles and abstracts for potentially eligible studies using Covidence. At title and abstract screening, we included studies in the full text review when the abstract reported both a physical activity and/or sedentary behavior outcome as well as a DHI engagement outcome (including meeting other inclusion and exclusion criteria). Therefore, studies that did not report a measure of association between physical activity and/or sedentary behavior and DHI engagement were still included for a full text review. This was done to ensure that the studies were not excluded in error. This screening process was implemented after an initial pilot screening of 100 full texts by MM, who found that none of the abstracts that reported only a health outcome or only an engagement outcome were incorrectly screened out. Following the title and abstract review, we obtained full texts of all potentially relevant or unclear articles, and authors (MM, TD, AG, and KR) independently reviewed these against our inclusion criteria. Reasons for exclusion were recorded in a characteristics of excluded studies table. The review authors were not blinded to author or journal information. The number of articles identified, screened, eligible, and included were recorded according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement [32] (Figure 1).

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram. *This review was conducted alongside another review aiming to describe the association between DHI engagement and dietary intake (PROSPERO CRD42018112189). Therefore, ‘dietary intake’ search terms were also included in the search strategy, but removed for reporting elsewhere.

 Data Extraction and Management

Pairs of review authors (MM and TD) independently extracted data using a data extraction form adapted from the Cochrane Public Health Group Methods Manual and used previously by the research team [33]. Given the complexity of the review, all the extracted data were reviewed by an experienced statistician and methods expert, and any disagreements were resolved by the third author (AH). In case of missing study data, we attempted to contact the corresponding authors to obtain the required information. The following data were extracted:

Study characteristics including authors’ name, year of publication, overall study design, intervention target (ie, intended intervention recipient), recruitment method (ecological or nonecological), and sample size.

Characteristics of the intervention, including type of DHI (ie, web-based, smartphone app, Exergame, and Facebook group), description of DHI components, and length of exposure to DHI.

Outcomes including both a description of the physical activity or sedentary behavior measure and the engagement measure; the analysis method used to examine associations including adjustments for confounding (for quality assessment); magnitude of the association (ie, odds ratio [OR] or regression coefficient or estimate along with a measure of variability [if available], 95% CIs or standard deviation or standard error); statistical significance of the association; and further information to allow quality assessment.

 Critical Appraisal

Pairs of review authors (MM, PM, and RS) assessed the methodological quality of studies independently using the Newcastle Ottawa Scale for cohort studies [34] (n=13 studies) and cross-sectional studies [35] (n=6 studies). We defined cross-sectional studies as the ones using a single time point of data for measuring the physical activity or sedentary behavior (eg, follow-up), whereas cohort studies as those that used multiple time points of data and calculated changes over time (eg, change from baseline to follow-up). All the studies were assessed based on their highest quality measure of association (ie, final follow-up time point, adjusted, device-measured physical activity or sedentary behavior, and objectively assessed engagement were preferred to mid-point follow-up, unadjusted, self-reported physical activity or sedentary behavior, and self-reported engagement).

The Newcastle Ottawa Scales utilizes a star system to assess the methodological quality of studies. The cohort tool assigns a maximum of 9 points for quality assessment in 3 domains: (1) selection of study groups (up to 4 points), (2) comparability of these groups (up to 2 points), and (3) assessment of outcomes (up to 3 points). The cross-sectional tool assigns a maximum of 10 points across the same 3 domains: (1) selection of study groups (up to 5 points), (2) comparability of these groups (up to 2 points), and (3) assessment of outcomes (up to 3 points; Multimedia Appendices 2 and 3 for scoring systems, adopted from Wells et al [34] and Modesti et al [35]).

Within the cohort tool, for the item “was follow up long enough for outcomes to occur,” studies were awarded a point if they had a minimum follow-up period of 3 months. This period was chosen based on the current evidence on the length of DHI engagement, where those that are designed to be used for 3 months or longer tend to be more effective than those designed for shorter durations [14,15]. Within the cross-sectional tool, for the item “the study controls for the most important factor (select one),” we selected age as the factor to control for, as it is unanimously possible to control for this across studies and is an important contextual factor influencing engagement [20]. Disagreements between assessments were resolved by discussion between the pairs of review authors (MM, RS, and PM) and, where required by consulting the third review author.

 Data Synthesis and Analysis

Data were synthesized according to the review objectives.

Objective 1

Describe the direction and strength of the association between engagement with DHIs and physical activity and/or sedentary behavior in adults.

We planned two separate meta-analyses: first for subjective experience and second for usage (including activities completed, time, and logins). However, we did not conduct a meta-analysis for subjective experience because of the considerable methodological heterogeneity between studies and the small number of studies reporting this outcome (n=3) [36]. Therefore, we focused on the direction and strength of the association between usage and physical activity or sedentary behavior.

A variety of different methods of association were used across the included studies. Consequently, to allow for meta-analysis, we were required to transform a number of estimates into one consistent effect index. A standardized regression coefficient was chosen as the effect index, which is a previously proposed method [37]. A list of the main transformations used for this analysis are detailed in Multimedia Appendix 4 and are predominantly based on the research works of Borenstein et al [36] and Nieminen et al [37].

We used the Dersimonian and Laird random effects method of meta-analysis to calculate a pooled standardized effect assessing the strength and direction of associations. Statistical analyses were performed using R [38]. Many studies have reported more than one association. For meta-analyses, we used the following hierarchical selection criteria to select a single association from each study for inclusion in the pooled synthesis:

Use measures were given preference in the following order:

activities completed, time on site then logins. This attribute reflects the level of participant involvement required for each measure of engagement, with a greater level of engagement given priority.

Measures of total physical activity were preferred to specific physical activity or sedentary behavior types (eg, measures of physical activity of moderate-to-vigorous intensity were preferred to distance walked).

Device-measured physical activity was preferred over self-report.

Whole-day measures of physical activity or sedentary behavior were preferred over specific time segments (eg, whole-day physical activity was preferred to workday physical activity).

Global scores of subjective experience engagement were preferred over the individual constructs of subjective experience (eg, engagement questionnaire score preferred to game-flow score only).

For DHIs with team or group engagement opportunities (eg, group challenges in a step-counting website), individual measures of engagement (eg, individual logins to the DHI) were preferred to group opportunities to engage (eg, total logins from a team of DHI users), as this more accurately reflects an individual’s DHI engagement.

Associations derived from fully adjusted models were preferred over unadjusted or partly adjusted models.

When studies did not provide sufficient data required for meta-analysis (ie, information to calculate an effect estimate and measure of variability of the effect estimate), the corresponding authors were contacted via email on up to 3 occasions and asked to provide information. When data were not available or not provided, we excluded the study from the meta-analysis. To provide supplementary data for this objective, we used vote-counting synthesis methods to describe the direction of association across these studies. We used the direction of association rather than statistical significance in accordance with the recent SWIM (Synthesis Without Meta-Analysis) guidelines [39]. We focused on a single measure of association from each study, which was selected based on the same hierarchical criteria for the selection of an association as for the meta-analysis. For vote counting, each study was summarized as either “+,” “−“ or “0.” “+” was assigned to the studies in which the point estimate and CI supported the hypothesis that higher engagement is associated with a higher physical activity or reduced sedentary behavior. “0” was assigned to the studies in which the point estimate and CI had inconclusive findings. “−“ was assigned to the studies in which the association point estimate and CI rejected the hypothesis. Several studies did not report any results, including an estimate or CI, but rather just stated whether the association was statistically significant in the hypothesized direction. In these instances, to ensure that such studies were included, we assigned either “+” or “−“ to the studies that reported significant association findings depending on the stated direction of the association (if provided) or “0” to the studies that reported nonsignificant findings.

 Objective 2

To explore whether the direction of association between engagement and physical activity or sedentary behavior varies according to the type of engagement (ie, subjective experience, activities completed, time, and logins).

To explore whether the direction or strength of associations varied between the different types of engagement, we classified each association across all the studies as either subjective experience, activities completed, time (time spent using the intervention; eg, session duration), or logins. Studies could contribute more than one association but only one association for each of the four types of engagement.

When studies had more than one association for a given type of engagement, we gave preference to the measures of association from adjusted associations and excluded the measures of association from the self-report measures of physical activity or sedentary behavior where device measures were available for the same engagement variable. Finally, for studies reporting associations at multiple time points, we included only data from the final time point.

We used vote-counting methods to explore the direction of the association between each type of engagement and physical activity or sedentary behavior outcomes. Each association was summarized as either “+,” “−,” or “0,” following the same procedures described above, with “0” being assigned to studies with associations reporting mixed findings.

 Results Search Results

The searches resulted in 13,192 potentially relevant abstracts. After removing the duplicates, 10,653 unique citations were retained for review. After the title and abstract screening, 375 full texts were identified and screened. Overall, 19 studies were included in our review (Figure 1).

 Study Characteristics

Detailed characteristics of each study (n=19) [22,24,40-56], and each measure of association, are provided in Multimedia Appendix 5. All 19 studies (n=7776 participants) were of physical activity measures, with no study using measures of sedentary behavior. Of the 19 studies, 12 were web-based interventions [22,24,41,44,45,47-50,52,54,55], 5 were app-based [43,46,51,53,56], and the remaining were Facebook-based (n=1) [42] and exergames (n=1) [40]. Cohort designs were used in 13 studies [22,24,42-44,47,48,50-52,54-56]. The remaining 6 studies used cross-sectional designs [40,41,45,46,49,53]. Across both cohort and cross-sectional studies, 11 studies included an analysis of the intervention arm of a randomized controlled trial [22,24,42-45,48,50,52,54,55].

The majority of studies included the usage measures of engagement (ie, activities completed, time, or logins; n=18), whereas 3 studies included subjective experience measures of engagement [40,46,48]. Participants across all the studies were predominately female (71%). Most studies used nonecological recruitment methods (n=11), with the remaining 8 studies using a mixture of ecological and nonecological recruitment methods [17,44,46,49,51,53,55,56]. The sample sizes across all the studies ranged from 7 to 3555 (mean 389; SD 760.6).

 Critical Appraisal (Quality Assessment)

Of the 19 studies, almost half were assessed to be of poor quality (n=9). Two studies were considered to be of fair quality, and the remaining studies (n=8) were considered to be of good quality. Quality assessment results for cohort studies are summarized in Table 1, and quality assessment results for cross-sectional studies are summarized in Table 2.

Quality assessment (Newcastle-Ottawa Quality Assessment Scale criteria for cohort studies).

Study Selection Comparability Outcome Selectiona
Representativeness of the exposed cohort Selection of the nonexposed cohort Ascertainment of exposure Outcome not present at the start of study Cohort statistical analysis Assessment of the outcome Was follow-up long enough for outcomes to occur Adequacy of follow-up cohorts
Carr et al [24] ★ ★ ★ ★ 0 0 ★ 0 Poor
Edney et al [42] ★ ★ ★ 0 0 0 0 ★ Poor
Edney et al [43] ★ ★ ★ ★ ★ ★ ★ ★ Good
Ferney et al [44] ★ ★ ★ 0 ★ 0 ★ ★ Good
Kwan et al [47] ★ ★ ★ 0 0 0 0 ★ Poor
Lewis et al [48] ★ ★ ★ ★ ★ 0 ★ ★ Good
Linke et al [50] ★ ★ ★ ★ ★ ★ ★ ★ Good
Ma et al [51] ★ ★ ★ 0 ★ ★ 0 ★ Good
Maher et al [52] ★ ★ ★ 0 ★ 0 0 ★ Poor
Marcus et al [22] ★ ★ ★ ★ ★ 0 ★ ★ Good
Rebar et al [54] ★ ★ ★ 0 ★★ 0 ★ 0 Poor
Wanner et al [55] ★ ★ ★ 0 ★ 0 ★ 0 Poor
Xian et al [56] ★ ★ ★ ★ ★ ★ 0 ★ Good

aQuality score: Overall scores were given (good, fair, and poor). Good quality: 3 or 4 stars (★) in the selection domain AND 1 or 2 stars in the comparability domain and 2 or 3 stars in the outcome domain; Fair quality: 2 stars in the selection domain and 1 or 2 stars in the comparability domain and 2 or 3 stars in the outcome/exposure domain; poor quality: 0 or 1 star in the selection domain OR 0 stars in the comparability domain OR 0 or 1 stars in the outcome/exposure domain.

Quality assessment (Newcastle-Ottawa Quality Assessment Scale criteria for cross-sectional studies).

Study Selection Comparability Outcome Quality scorea
Representativeness of the exposed cohort Sample size Comparability of nonrespondents Ascertainment of the exposure Statistical analysis design features Assessment of outcome Statistical test
Bronner et al [40] 0 ★ 0 ★ 0 ★★ 0 Poor
Davies et al [41] ★ 0 ★ ★ ★★ ★ 0 Poor
Hansen et al [45] ★ ★ ★ ★ 0 ★ 0 Poor
Hoj et al [46] ★ 0 0 ★ ★ ★ ★ Fair
Lieber et al [49] ★ 0 ★ ★ ★ ★ ★ Good
Marquet et al [53] ★ 0 0 ★ ★ ★★ ★ Fair

aQuality score: Overall scores were given (good, fair, and poor). Good quality: 3 or 4 stars (★) in the selection domain AND 1 or 2 stars in the comparability domain and 2 or 3 stars in the outcome domain; fair quality: 2 stars in the selection domain and 1 or 2 stars in the comparability domain and 2 or 3 stars in the outcome/exposure domain; poor quality: 0 or 1 star in the selection domain OR 0 stars in the comparability domain OR 0 or 1 stars in the outcome/exposure domain.

 Selection

Within the cohort studies (n=13), all studies scored highly in the selection domain. Representativeness of the sample was high, with all studies scoring a star for being either truly or somewhat representative of the average target population. The nonexposed cohort was drawn from the same community as the exposed cohort in all studies. The exposure (engagement) was usually measured using either objective measurement (eg, Google Analytics) or self-report.

Within cross-sectional studies (n=6), all but one study had somewhat representative or truly representative samples. Sample size calculations were often not provided (n=4). Nonresponse characteristics were not provided or poorly described in half of the studies. No studies used validated measurement tools; however, the tool was made available or well described in all studies.

 Comparability

Cohort studies controlled for confounders in 10 of the 13 studies. However, only one study controlled for all 3 factors (ie, age, sex, and marital status) required to score two stars. Therefore, most studies scored one star. Three studies scored zero stars, as they used unadjusted analyses.

In the cross-sectional studies, only 2 studies used adjusted analyses. One further study controlled for age and scored an additional star.

 Outcome

Within the cohort studies, four studies used an independent blind assessment or record linkage (eg, steps via a mobile phone). The remaining studies scored zero stars as they used self-reporting. Eight studies were followed up after a sufficient duration (3 months); therefore, they scored a star. Five studies had follow-up shorter than 3 months. The follow-up cohort rate was inadequate in 3 studies, as no description of differences in responders and nonresponders was provided, and less than 80% responded. The remaining 10 studies scored a star, as either more than 80% responded at follow-up, or there were no differences in responders and nonresponders.

Of the cross-sectional studies, most studies used self-reported physical activity measurements (n=4). Half of the studies were considered to have used appropriate and well-described statistical tests; the remaining studies did not describe or provide sufficient details (eg, measures of variance).

 Objective 1

Describe the direction and strength of the association between engagement with DHIs and physical activity or sedentary behavior in adults.

Although we had planned two meta-analyses, one for each of the conceptually different forms of engagement (use and subjective experience) [20], we did not conduct a meta-analysis for subjective experience because of the considerable methodological heterogeneity among the studies and the small number of studies reporting this outcome (n=3) [36]. Therefore, for this objective, we focused on the direction and strength of the association between usage and physical activity. There were 18 studies reporting a usage outcome, of which 7 were excluded from the meta-analysis [22,24,45,46,52,54,56], as data were not available to allow the calculation of an effect estimate or a measure of variability of the effect estimate data were not available.

The results from the meta-analysis of usage associations (n=11 studies) are shown in Figure 2 [41-44,47-51,53,55]. The characteristics of each of the associations from the meta-analysis are included in Table 3. The pooled estimate of the standardized regression coefficient (0.08; 95% CI 0.01-0.14; P=.02; SD 0.11) indicated a small but significant positive relationship between engagement with a DHI and physical activity. Heterogeneity was high, with 77% of the variation in the point estimates explained by the between-study heterogeneity.

Meta-analysis results from 11 studies to assess the direction and strength of the relationship between engagement with a digital health intervention and physical activity using the Dersimonian and Laird method.

Characteristics of studies (n=11) included in meta-analysisa.

Study DHIb type Engagement measure Physical activity measure
Edney et al [42] Facebook Activities completed MVPAc
Ma et al [51] Smartphone app Time Distance travelled
Lieber et al [49] Web-based Activities completed MVPA
Wanner et al [55] Web-based Time MVPA
Ferney et al [44] Web-based Logins MVPA
Lewis et al [48] Web-based Activities completed MVPA
Kwan et al [47] Web-based Logins MVPA
Marquet et al [53] Smartphone app Time Steps
Linke et al [50] Web-based Time MVPA
Edney et al [43] Smartphone app Activities completed MVPA
Davies et al [41] Web-based Activities completed Steps

aA single association was selected for each study based on hierarchical criteria, see methods section. See Multimedia Appendix 5 for full details.

bDHI: digital health intervention.

cMVPA: moderate to vigorous physical activity.

The findings of the 7 studies that could not be included in the meta-analysis are summarized in Table 4. Four of the 7 studies reported an association consistent with the hypothesis that higher engagement (usage measures) is associated with a higher physical activity [22,45,52,56]. The remaining 3 studies had inconclusive findings [24,46,54]. Two studies had focused on the activities completed [24,56], one on time [54], and the remaining 4 on logins [22,45,46,52]. The inconclusive findings [24,46,54] were from the associations of different constructs of engagement (ie, activities completed, time, and logins).

Characteristics and vote counting of studies (n=7) not included in meta-analysis.

Study DHIa type Engagement measure Physical activity measure Association type Association Directionb
Carr et al [24] Web-based Activities completed Steps Multiple linear regression Text only: “No other predictors,...[including engagement]...significantly explained...change in physical activity.” 0
Hansen et al [45] Web-based Logins MVPAc Kruskal-Wallis test P≤.001 +
Hoj et al [46] Smartphone app Logins Physical activity score Multiple regression SE −0.01 (0.067) 0
Maher et al [52] Web-based Logins MVPA Generalized linear mixed models (F1,41=3.06; P=.04) +
Marcus et al [22] Web-based Logins MVPA Quantile regression β=34.32 (95% CI 14.33 to 54.31) +
Rebar et al [54] Web-based Time MVPA Linear mixed models γ=0.51 (95% CI −1.77 to 2.72); P>.05 0
Xian et al [56] Smartphone app Activities completed Steps Ordinal least squares regression Every 10,000 XP points gained were associated with 2134 additional steps per day (95% CI 1673 to 2595; P<.001; R2=0.33]) +

aDHI: digital health intervention.

bA single association was selected for each study based on hierarchical criteria, see Methods section. Each study was summarized as either “+,” “−,” or “0.” “+” was assigned to the studies in which the point estimate and CI supported the hypothesis that higher engagement is associated with higher physical activity or reduced sedentary behavior. “0” was assigned to studies in which the point estimate and CI had inconclusive findings. “−“ was assigned to studies where the association point estimate and CI rejected the hypothesis. We assigned either “+” or “−“ to the studies without point estimates or CIs that reported significant association findings. We assigned “0” to the studies without point estimates or CIs that reported nonsignificant findings. See Multimedia Appendix 5 and Methods section for full details.

cMVPA: moderate to vigorous physical activity.

 Objective 2

Explore whether the direction of association between engagement and physical activity or sedentary behavior varies with the type of engagement (ie, subjective experience, activities completed, time, and logins).

Studies measured associations between physical activity and subjective experience (n=3) [40,46,48], activities completed (n=8) [24,41-43,48-50,56], time (n=5) [50,51,53-55], and logins (n=10) [22,24,44-48,50,52,54]. Therefore, 26 measures of association were included. The results of vote counting are summarized in Table 5. Overall, most associations (15 of 26) were in the hypothesized direction, stating that higher engagement is associated with higher physical activity. One association rejected the hypothesized direction, and the remaining 10 associations had inconclusive findings.

For the three domains of engagement, the direction consistently supported the hypothesis: subjective experience (2 of 3) [46,48], activities completed (5 of 8) [43,48-50,56], and logins (6 of 10) [22,45,48,50,52,54]. However, for time (n=5 associations), the findings did not support a positive association consistently, 2 studies had inconclusive findings [54,55], and one association rejected the hypothesized direction [51].

The 3 studies that described an association between subjective experience and physical activity used different measures to assess subjective experience. Bronner et al [40] used an Exergame Questionnaire that included questions similar to previously validated questions to assess subjective experience engagement in video games. The Exergame Questionnaire contained separate sections for engagement, game flow, and usability. The length of the questionnaire was not clear. Hoj et al [46] devised a five-question subjective experience questionnaire and constructed a composite score from it. Finally, Lewis et al [48] used the five-item Website Quality Questionnaire and constructed a composite score.

Summary of associations included in vote counting.

Study Engagement measure DHIa type Association type Association Directionb
Bronner et al [40] Subjective experience Exergame Pearson’s correlation 3 Associations:

ρ=0.61

ρ=0.52

Not reported

 0
Hoj et al [46] Subjective experience Smartphone app Multiple regression SE 0.40 (0.074) +
Lewis et al [48] Subjective experience Web-based Quintile regression t=2.32 (P≤.01) +
Carr et al [24] Activities completed Web-based Multiple regression Not reported (nonsignificant) 0
Davies et al [41] Activities completed Web-based Odds ratio 3 Associations:

ORc 2.80 (95% CI 1.45 to 5.40)

Not reported (nonsignificant)

Not reported (nonsignificant)

 0
Edney et al [42] Activities completed Facebook group Pearson’s correlation ρ=−0.13 0
Edney et al [43] Activities completed Smartphone app Linear mixed models F1,272=4.5 (P=.04) +
Lewis et al [48] Activities completed Web-based Odds ratio OR 1.29 (95% CI 1.14 to 1.47) +
Lieber et al [49] Activities completed Web-based Odds ratio OR 1.05 (95% CI 1.01 to 1.09) +
Linke et al [50] Activities completed Web-based Generalized linear models 3 Associations:

β=2.85; SE: 1.38 (P=.04)

β=1.00; SE: 0.82 (P=.05)

β=3.49; SE: 1.28 (P=.01)

 +
Xian et al [56] Activities completed Smartphone app Ordinal least squares regression Every 10,000 XP were associated with 2134 additional steps per day (95% CI 1673 to 2595; P<.001; R2=0.33) +
Linke et al [50] Time Web-based Generalized linear models β=0.48, SE: 0.20; P=.02 +
Ma et al [51] Time Smartphone app Multi-level modelling β=−0.005; P≤.001 −
Marquet et al [53] Time Smartphone app ANCOVA ρ=0.176; P<.05 +
Rebar et al [54] Time Web-based Linear mixed models 2 associations:

γ=2.33 (95% CI 0.09 to 4.64); P<.05

γ=0.51 (95% CI −1.77 to 2.72); P>.05

 0
Wanner et al [55] Time Web-based Linear regression 95% CI 0.58 (−0.43 to 1.59; P=.26) 0
Carr et al [24] Logins Web-based Multiple regression Not reported (nonsignificant) 0
Ferney et al [44] Logins Web-based ANCOVA 4 Associations

P=.69

P=.70

P=.09

P=.05

 0
Hansen et al [45] Logins Web-based Kruskal-Wallis test P≤.001 +
Hoj et al [46] Logins Smartphone app Multiple regression SE −0.01 (0.067) 0
Kwan et al [47] Logins Web-based ANOVAd F1,63=1.54, P=.22, np2=0.03 0
Lewis et al [48] Logins Web-based Quintile regression t=3.39 (P≤.01) +
Linke et al [50] Logins Web-based Generalized linear models Not reported (nonsignificant) +
Maher et al [52] Logins Web-based Generalized linear mixed models F1,41=3.06 (P=.04) +
Marcus et al [22] Logins Web-based Quantile regression β=34.32 (95% CI 14.33 to 54.31) +
Rebar et al [54] Logins Web-based Linear mixed models 2 Associations:

γ=3.18 (95% CI 1.15 to 5.07); P<.05

γ=2.04 (95% CI 0.29 to 3.84); P<.05

 +

aDHI: digital health intervention.

b”+,” “−,” or “0” were assigned. “+” was assigned to studies where all associations within the particular engagement domain (subjective experience, activities completed, time and logins) where the point estimates and CIs supported the hypothesis that higher engagement is associated with higher physical activity or reduced sedentary behavior. “0” was assigned to the studies with inconclusive or mixed associations. “−“ was assigned to the studies where all point estimates and CIs rejected the hypothesis that higher engagement is associated with higher physical activity or reduced sedentary behavior. See Multimedia Appendix 5 and Methods section for full details.

cOR: odds ratio.

dANOVA: analysis of variance.

 Discussion Principal Findings

The findings of this review suggest that there is a positive relationship between engagement with a physical activity, both objective usage and subjective experience, and physical activity outcomes in adults. The strength of the relationship between DHI usage and physical activity based on a meta-analysis of 11 studies is weak (0.08, 95% CI 0.01-0.14). The direction of the association between physical activity and engagement was consistent across different measures of engagement, including two measures of usage (activities completed and logins) and subjective experience, but was less clear for the third measure of usage—time (ie, session duration). The majority of associations for subjective experience, activities completed, and logins were positive, whereas the remainder were inconclusive. There was a mixture of positive, inconclusive, and a negative association. No studies have examined the relationship between DHI engagement and sedentary behavior outcomes.

 Findings in Context

This review updates by 10 years and expands on the review by Donkin et al [8], which identified 3 studies assessing the association between the usage of physical activity DHIs and physical activity outcomes [8]. In agreement with our review, Donkin et al [8] reported a consistent positive relationship between usage outcomes (eg, logins and activities completed) and DHIs targeting physical health (ie, psychological health, dietary behavior, physical activity, weight management, and smoking; 31/33 studies) [8]. Logins and activities completed were the most common engagement outcomes included in both the reviews. Donkin et al [8] did not include any studies with associations between time and physical health behavior, whereas our review contributes 5 studies [50,51,53-55]. Our findings for time were inconsistent, which aligns with the nonhealth studies exploring user engagement with internet-based news websites, which have found that time is not a reliable indicator of engagement [57].

We found a positive but weak relationship between DHI usage and physical activity. In contrast to the usage, it has been suggested that a clearer dose-response relationship exists between subjective experience engagement (eg, how captivating of attention a DHI is, the emotions a DHI elicits, and how interesting participants find a DHI) and effectiveness [20]. This further highlights the importance of defining the types of engagement outcomes as well as using multiple indicators of engagement when trying to understand the relationship between engagement variables and the effectiveness of DHIs [58,59].

This is the first review to examine the relationship between DHI subjective experience engagement and physical activity. Only 3 studies have reported associations between subjective experience and physical activity, with 2 of the associations in the hypothesized direction [40,46,48]. Each study used different self-reporting tools to assess different constructs of subjective experience (ie, website usefulness, app likeability, engagement, game-flow, and usability), which has previously been identified as an issue in the assessment of subjective experience engagement [60]. Such heterogeneity in constructs makes comparisons difficult, even though the direction of association is consistently positive. Future studies should focus on using consistent measures of subjective experience that are valid and reliable to enable comparisons between studies [61,62].

Further research into the relationship between subjective experience engagement and usage engagement is also warranted, as some qualitative studies suggest that usage is positively related to subjective experience [63,64]. For example, results from interviews with participants involved in an internet-based physical activity intervention reported that usage was positively influenced by subjective experience factors (eg, trust, reliability, and functionality of the program) [64]. Another study found that the sustained use of a Fitbit activity tracker was influenced by subjective experience–related factors (eg, empty batteries, broken trackers, and user experience) [63]. In other health behaviors, people who smoke and consume alcohol who wish to quit or cut down have suggested that the look, feel, app store quality rating, branding, and wording of the title are important while choosing or not choosing to use an app [65]. Therefore, improving subjective experience could increase the strength of the positive relationship between usage and physical activity outcomes found in this review.

 Strengths

A key strength of this review was the focus on two health behaviors (physical activity and sedentary behavior), reducing the heterogeneity and increasing the validity of findings [20]. This is the first review to examine the association between the subjective experience of DHI and health behavior. Including subjective experience and usage recognizes that engagement goes beyond usage, while considering attention, interest, and affect [20]. Excluding DHIs targeting individuals with a specific health condition reduces heterogeneity, as the context (including population) is known to influence engagement [20]. Another strength is the use of meta-analyses to examine the strength of the association between DHI usage and physical activity. Although it was not possible to conduct separate meta-analyses within engagement constructs, the use of vote-counting methods to assess the direction of association is a recommended method when meta-analyses are not possible [39].

 Limitations

This review should be interpreted in the context of its limitations. The first limitation was that we included many different study types that produced large heterogeneity in the included studies. It means that we had to transform effect estimates to a common effect estimate and combine standardized effects, making interpretation of the results difficult. Furthermore, 8 studies were excluded from our meta-analysis because they provided insufficient information to be included in the meta-analysis. For vote counting, where studies did not report point estimates and CIs, we had to rely on the wording provided by the authors to infer whether the results supported our hypothesis. Second, the analysis of the association between engagement outcomes and physical activity outcomes in all 19 studies was a secondary analysis for these studies. Such analyses were often not well described. A further limitation was the inclusion of all recruitment types (ie, ecological and nonecological) within the same meta-analysis and vote-counting (ie, ecological and nonecological). It is known that ecological recruitment methods lead to higher attrition and lower engagement [31]. For example, Wanner et al [55] and Vandelanotte et al [31] highlight that spontaneous users (ecological) report a much lower engagement and higher dropout, whereas those that remain engaged become as active as those in the randomized groups (nonecological groups), possibly due to differing motivations [31,55]. In addition, although our search methods were rigorous, it is possible that expanding the search databases to include the ACM Digital Library may have identified additional studies from the human-computer interaction literature. Finally, given the lack of quantitative studies on subjective experience, perhaps owing to subjective experience being more often measured qualitatively [20], we encourage future reviews to explore the relationship between engagement and physical activity and sedentary behavior in qualitative studies.

 Conclusions

A weak but consistent positive relationship exists between engagement with a physical activity DHI and physical activity outcomes. This is consistent across 2 of the 3 indicators of usage engagement that we examined, and subjective experience engagement; however, there are weak effect sizes. A further exploration of the relationship between engagement and physical activity using valid and reliable measurement tools is warranted, given the heterogeneity in measurement tools. Additional focus should be directed at DHI subjective experience (ie, attention, interest, and affect) by using consistent methodology to explore its relationship with the usage of DHIs and health behavior outcomes. Given the absence of studies, further research examining the association between DHIs and the impact on sedentary behavior is also warranted.

Search strategy.

Newcastle Ottawa Scale for cohort studies.

Newcastle Ottawa Scale for cross-sectional studies.

Data transformations.

Supplementary table of characteristics of included studies.

 Abbreviations DHI

digital health intervention

 PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

 PROSPERO

International Prospective Register for Systematic Reviews

The authors convey sincere thanks to Debbie Booth, who was the Senior Librarian, for conducting the searches across scientific databases.

MM drafted the manuscript. MM, TD, and LW developed the concept for the review. MM, TD, and JB screened all abstracts. AG, KR, MM, and TD screened all full texts. MM, TD, and AH extracted all data. MM, PM, and RS conducted the quality assessment. All authors contributed to the methods for the review, multiple versions of the manuscript, and approved the final version of the manuscript.

None declared.

 Ekelund U Tarp J Steene-Johannessen J Hansen BH Jefferis B Fagerland MW Whincup P Diaz KM Hooker SP Chernofsky A Larson MG Spartano N Vasan RS Dohrn I Hagströmer M Edwardson C Yates T Shiroma E Anderssen SA Lee I Dose-response associations between accelerometry measured physical activity and sedentary time and all cause mortality: systematic review and harmonised meta-analysis BMJ 2019 08 21 l4570 10.1136/bmj.l4570 Biswas A Oh PI Faulkner GE Bajaj RR Silver MA Mitchell MS Alter DA Sedentary Time and Its Association With Risk for Disease Incidence, Mortality, and Hospitalization in Adults Ann Intern Med 2015 01 20 162 2 123 10.7326/m14-1651 Pandey A Salahuddin U Garg S Ayers C Kulinski J Anand V Mayo H Kumbhani DJ de Lemos J Berry JD Continuous Dose-Response Association Between Sedentary Time and Risk for Cardiovascular Disease JAMA Cardiol 2016 08 01 1 5 575 10.1001/jamacardio.2016.1567 World Health Organization Global action plan on physical activity 2018-2030: more active people for a healthier world Noncommunicable diseases and their risk factors 2018 Geneva World Health Organization Iribarren SJ Cato K Falzon L Stone PW What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions PLoS One 2017 2 2 12 2 e0170581 10.1371/journal.pone.0170581 28152012 PONE-D-16-26479 PMC5289471 International Telecommunication Union ICT Facts and Figures 2016 Mobile network coverage and evolving technologies 2016 2016 WHO Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment - 2016 1 144 10.1017/cbo9780511712074.003 Donkin L Christensen H Naismith SL Neal B Hickie IB Glozier N A systematic review of the impact of adherence on the effectiveness of e-therapies J Med Internet Res 2011 08 05 13 3 e52 10.2196/jmir.1772 21821503 v13i3e52 PMC3222162 World HO WHO guideline: recommendations on digital interventions for health system strengthening Sexual and reproductive health 2019 Geneva World Health Organization Müller AM Maher CA Vandelanotte C Hingle M Middelweerd A Lopez ML DeSmet A Short CE Nathan N Hutchesson MJ Poppe L Woods CB Williams SL Wark PA Physical Activity, Sedentary Behavior, and Diet-Related eHealth and mHealth Research: Bibliometric Analysis J Med Internet Res 2018 04 18 20 4 e122 10.2196/jmir.8954 29669703 v20i4e122 PMC5932335 WHO mHealth: new horizons for health through mobile technologies Global Observatory for eHealth series - Volume 3 2011 Geneva World Health Organanization Müller Andre Matthias Alley S Schoeppe S Vandelanotte C Int J Behav Nutr Phys Act 2016 10 10 13 1 109 10.1186/s12966-016-0434-2 27724911 10.1186/s12966-016-0434-2 PMC5057225 Maher C Lewis L Ferrar K Marshall S De Bourdeaudhuij Ilse Vandelanotte C Are health behavior change interventions that use online social networks effective? A systematic review J Med Internet Res 2014 02 14 16 2 e40 10.2196/jmir.2952 24550083 v16i2e40 PMC3936265 Davies C Spence J Vandelanotte C Caperchione C Mummery W Meta-analysis of internet-delivered interventions to increase physical activity levels Int J Behav Nutr Phys Act 2012 04 30 9 52 10.1186/1479-5868-9-52 22546283 1479-5868-9-52 PMC3464872 Jahangiry L Farhangi M Shab-Bidar S Rezaei F Pashaei T Web-based physical activity interventions: a systematic review and meta-analysis of randomized controlled trials Public Health 2017 152 a Hou S Charlery SR Roberson K Systematic literature review of Internet interventions across health behaviors Health Psychol Behav Med 2014 01 01 2 1 455 481 10.1080/21642850.2014.895368 25750795 895368 PMC4345904 Romeo A Edney S Plotnikoff R Curtis R Ryan J Sanders I Crozier A Maher C Can Smartphone Apps Increase Physical Activity? Systematic Review and Meta-Analysis J Med Internet Res 2019 03 19 21 3 e12053 10.2196/12053 30888321 v21i3e12053 PMC6444212 Bort-Roig J Gilson N Puig-Ribera A Contreras R Trost S Measuring and influencing physical activity with smartphone technology: a systematic review Sports Med 2014 05 44 5 671 86 10.1007/s40279-014-0142-5 24497157 Flores Mateo Gemma Granado-Font E Ferré-Grau Carme Montaña-Carreras Xavier Mobile Phone Apps to Promote Weight Loss and Increase Physical Activity: A Systematic Review and Meta-Analysis J Med Internet Res 2015 11 10 17 11 e253 10.2196/jmir.4836 26554314 v17i11e253 PMC4704965 Perski O Blandford A West R Michie S Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis Transl Behav Med 2017 06 7 2 254 267 10.1007/s13142-016-0453-1 27966189 10.1007/s13142-016-0453-1 PMC5526809 McKay H King D Eakin E Seeley J Glasgow R The diabetes network internet-based physical activity intervention: a randomized pilot study Diabetes Care 2001 08 24 8 1328 34 10.2337/diacare.24.8.1328 11473065 Marcus B Lewis B Williams D Dunsiger S Jakicic J Whiteley J A Comparison of Internet and Print-Based Physical Activity Interventions JAMA Internal Medicine 2007 167 9 944 9 17502536 Carr LJ Bartee RT Dorozynski CM Broomfield JF Smith ML Smith DT Eight-month follow-up of physical activity and central adiposity: results from an Internet-delivered randomized control trial intervention J Phys Act Health 2009 07 6 4 444 55 10.1123/jpah.6.4.444 19842458 PMC2820723 Carr LJ Bartee RT Dorozynski C Broomfield JF Smith ML Smith DT Internet-delivered behavior change program increases physical activity and improves cardiometabolic disease risk factors in sedentary adults: results of a randomized controlled trial Prev Med 2008 05 46 5 431 8 10.1016/j.ypmed.2007.12.005 18207228 S0091-7435(07)00503-8 Short C Rebar A Plotnikoff R Vandelanotte C Designing engaging online behaviour change interventions a proposed model of user engagement Netherlands urpean Health Psychology Society 2015 E Moola S Munn Z Sears K Sfetcu R Currie M Lisy K Conducting systematic reviews of association (etiology): The Joanna Briggs Institute's approach International journal of evidence-based healthcare 2015 13 3 163 9 26262566 Tessa D Mclaughlin M Yoong S Wyse R Sutherland R Hollis J Brown A Barnes C Dray J Wiggers J Wolfenden L Associations between digital health intervention engagement and dietary intake: a systematic review 2018 10 2189 10.13140/RG.2.2.32035.71203 Dobbins M Husson H DeCorby K LaRocca R School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6 to 18 The Cochrane database of systematic reviews(2):Cd00 2013 7651 Alkhaldi G Hamilton F Lau R Webster R Michie S Murray E The effectiveness of prompts to promote engagement with digital interventions: a systematic review Journal of medical Internet research 2016 18 1 a Milward J Drummond C Fincham-Campbell S Deluca P What makes online substance-use interventions engaging? A systematic review and narrative synthesis Digit Health 2018 4 2055207617743354 10.1177/2055207617743354 29942622 10.1177_2055207617743354 PMC6001270 Vandelanotte C Duncan MJ Kolt GS Caperchione CM Savage TN Van Itallie A Oldmeadow C Alley SJ Tague R Maeder AJ Rosenkranz RR Mummery WK More real-world trials are needed to establish if web-based physical activity interventions are effective Br J Sports Med 2019 12 53 24 1553 1554 10.1136/bjsports-2018-099437 29970409 bjsports-2018-099437 Moher D Liberati A Tetzlaff J Altman DG PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement Ann Intern Med 2009 08 18 151 4 264 9, W64 10.7326/0003-4819-151-4-200908180-00135 19622511 0000605-200908180-00135 Wolfenden L Goldman S Stacey F Grady A Kingsland M Williams C Wiggers John Milat Andrew Rissel Chris Bauman Adrian Farrell Margaret M Légaré France Ben Charif Ali Zomahoun Hervé Tchala Vignon Hodder Rebecca K Jones Jannah Booth Debbie Parmenter Benjamin Regan Tim Yoong Sze Lin Strategies to improve the implementation of workplace-based policies or practices targeting tobacco, alcohol, diet, physical activity and obesity Cochrane Database Syst Rev 2018 11 14 11 CD012439 10.1002/14651858.CD012439.pub2 30480770 PMC6362433 Wells G Shea B O'Connell J The Newcastle-Ottawa Scale (NOS) for assessing the quality of non randomised studies in meta-analyses Ohrica 2014 2020-12-23 https://www.researchgate.net/publication/288802810_The_Newcastle-Ottawa_Scale_NOS_for_Assessing_The_Quality_of_Nonrandomised_Studies_in_Meta-analyses Modesti P Reboldi G Cappuccio F Agyemang C Remuzzi G Rapi S Perruolo Eleonora Parati Gianfranco ESH Working Group on CV Risk in Low Resource Settings Panethnic Differences in Blood Pressure in Europe: A Systematic Review and Meta-Analysis PLoS One 2016 11 1 e0147601 10.1371/journal.pone.0147601 26808317 PONE-D-15-03808 PMC4725677 Michael BLJ When Does it Make Sense to Perform a Meta‐Analysis? Chapter 40 2009 2020 Nieminen P Lehtiniemi H Vähäkangas K Huusko A Rautio A Standardised regression coefficient as an effect size index in summarising findings in epidemiological studies Epidemiology, Biostatistics and Public Health 2013 10 - 10.2427/8854 A language and environment for statistical computing R Foundation for Statistical Computing, Vienna, Austria 2020-12-23 http://www.R-project.org/ Campbell M McKenzie JE Sowden A Katikireddi SV Brennan SE Ellis S Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline BMJ 2020 368 - Bronner S Pinsker R Adam NJ Physiological and psychophysiological responses in experienced players while playing different dance exer-games Computers in Human Behavior 2015 51 34 41 Davies C Corry K Van Itallie Anetta Vandelanotte C Caperchione C Mummery W Prospective associations between intervention components and website engagement in a publicly available physical activity website: the case of 10,000 Steps Australia J Med Internet Res 2012 01 11 14 1 e4 10.2196/jmir.1792 22260810 v14i1e4 PMC3846338 Edney S Looyestyn J Ryan J Kernot J Maher C Posts, pics, or polls? Which post type generates the greatest engagement in a Facebook physical activity intervention? Transl Behav Med 2018 11 21 8 6 953 957 10.1093/tbm/iby006 29660093 4961537 Edney S Ryan JC Olds T Monroe C Fraysse F Vandelanotte C Plotnikoff R Curtis R Maher C User Engagement and Attrition in an App-Based Physical Activity Intervention: Secondary Analysis of a Randomized Controlled Trial J Med Internet Res 2019 11 27 21 11 e14645 10.2196/14645 31774402 v21i11e14645 PMC6906621 Ferney SL Marshall AL Eakin EG Owen N Randomized trial of a neighborhood environment-focused physical activity website intervention Prev Med 2009 02 48 2 144 50 10.1016/j.ypmed.2008.10.022 19028519 S0091-7435(08)00574-4 Hansen A Grønbæk M Helge J Severin M Curtis T Tolstrup J Effect of a Web-Based Intervention to Promote Physical Activity and Improve Health Among Physically Inactive Adults: A Population-Based Randomized Controlled Trial J Med Internet Res 2012 14 5 A Hoj T Covey E Jones A Haines A Hall P Crookston B How Do Apps Work? An Analysis of Physical Activity App Users? Perceptions of Behavior Change Mechanisms JMIR Mhealth Uhealth 2017 5 8 e114 28778846 Kwan M Faulkner G Bray S Evaluation of active transition, a website-delivered physical activity intervention for university students: pilot study JMIR Res Protoc 2013 04 29 2 1 e16 10.2196/resprot.2099 23649858 v2i1e16 PMC3650928 Lewis B Williams D Dunsiger S Sciamanna C Whiteley J Napolitano M Bock B Jakicic J Getz M Marcus B User attitudes towards physical activity websites in a randomized controlled trial Prev Med 2008 11 47 5 508 13 10.1016/j.ypmed.2008.07.020 18773915 S0091-7435(08)00400-3 PMC2614822 Lieber SB Redberg RF Blumenthal RS Gandhi A Robb KJ Mora S A national interactive web-based physical activity intervention in women, evaluation of the american heart association choose to move program 2006-2007 Am J Cardiol 2012 06 15 109 12 1754 60 10.1016/j.amjcard.2012.02.017 22494850 S0002-9149(12)00693-5 Linke SE Dunsiger SI Gans KM Hartman SJ Pekmezi D Larsen BA Mendoza-Vasconez AS Marcus BH Association Between Physical Activity Intervention Website Use and Physical Activity Levels Among Spanish-Speaking Latinas: Randomized Controlled Trial J Med Internet Res 2019 07 24 21 7 e13063 10.2196/13063 Ma BD Ng SL Schwanen T Zacharias J Zhou M Kawachi I Sun G Pokémon GO and Physical Activity in Asia: Multilevel Study J Med Internet Res 2018 06 15 20 6 e217 10.2196/jmir.9670 Maher C Ferguson M Vandelanotte C Plotnikoff R De Bourdeaudhuij I Thomas S Nelson-Field K Olds T A Web-Based, Social Networking Physical Activity Intervention for Insufficiently Active Adults Delivered via Facebook App: Randomized Controlled Trial J Med Internet Res 2015 07 13 17 7 e174 10.2196/jmir.4086 26169067 v17i7e174 PMC4526990 Marquet O Alberico C Hipp A Pokémon GO and physical activity among college students. A study using Ecological Momentary Assessment Computers in Human Behavior 2018 04 81 215 222 10.1016/j.chb.2017.12.028 Rebar A Boles CW W. Burton N Duncan Mj Short Ce Happell B Kolt Gs Caperchione Cm Rosenkranz Rr Vandelanotte C Healthy mind, healthy body: A randomized trial testing the efficacy of a computer-tailored vs. interactive web-based intervention for increasing physical activity and reducing depressive symptoms Mental Health and Physical Activity 2016 10 11 29 37 10.1016/j.mhpa.2016.08.001 Wanner M Martin-Diener E Braun-Fahrländer C Bauer G Martin BW Effectiveness of active-online, an individually tailored physical activity intervention, in a real-life setting: randomized controlled trial J Med Internet Res 2009 07 28 11 3 e23 10.2196/jmir.1179 19666456 v11i3e23 PMC2763402 Xian Y Xu H Xu H Liang L Hernandez AF Wang TY Peterson ED An Initial Evaluation of the Impact of Pokémon GO on Physical Activity J Am Heart Assoc 2017 05 16 6 5 - 10.1161/JAHA.116.005341 28512111 JAHA.116.005341 PMC5524088 O'Brien H Lebow M Mixed-methods approach to measuring user experience in online news interactions J Am Soc Inf Sci Tec 2013 06 19 64 8 1543 1556 10.1002/asi.22871 Michie S Yardley L West Robert Patrick Kevin Greaves Felix Developing and Evaluating Digital Interventions to Promote Behavior Change in Health and Health Care: Recommendations Resulting From an International Workshop J Med Internet Res 2017 06 29 19 6 e232 10.2196/jmir.7126 28663162 v19i6e232 PMC5509948 Cole-Lewis H Ezeanochie N Turgiss J Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions JMIR Form Res 2019 10 10 3 4 e14052 10.2196/14052 31603427 v3i4e14052 PMC6813486 Ng M Firth J Minen M Torous J User Engagement in Mental Health Apps: A Review of Measurement, Reporting, and Validity Psychiatr Serv 2019 07 01 70 7 538 544 10.1176/appi.ps.201800519 30914003 PMC6839109 Brooke J SUS -- a quick and dirty usability scale - 1996 Oxfordshire, United Kingdom Taylor & Francis A O’Brien Hl Cairns P Hall M A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form International Journal of Human-Computer Studies 2018 04 112 28 39 10.1016/j.ijhcs.2018.01.004 Hermsen S Moons J Kerkhof P Wiekens C De Groot M Determinants for Sustained Use of an Activity Tracker: Observational Study JMIR Mhealth Uhealth 2017 10 30 5 10 e164 10.2196/mhealth.7311 29084709 v5i10e164 PMC5695980 Bossen D Buskermolen M Veenhof C de Bakker Dinny Dekker J Adherence to a web-based physical activity intervention for patients with knee and/or hip osteoarthritis: a mixed method study J Med Internet Res 2013 10 16 15 10 e223 10.2196/jmir.2742 24132044 v15i10e223 PMC3806355 Perski O Blandford A Ubhi H West R Michie S Smokers' and drinkers' choice of smartphone applications and expectations of engagement: a think aloud and interview study BMC Med Inform Decis Mak 2017 02 28 17 1 25 10.1186/s12911-017-0422-8 28241759 10.1186/s12911-017-0422-8 PMC5329928