The electronic databases Medline, PsycINFO, Embase, CINAHL, and Scopus were searched to retrieve studies describing the effectiveness of internet-based interventions on 2 or more SNAP behaviors published by November 18, 2019. The following combinations of keywords were used: (multiple health behavio* or multiple behavio* or multiple risk* or multiple health* or smok* or tobacco or alcohol or diet* or nutrition or exercise or physical activity or fruit* or vegetable*) AND (internet or web* or online or on-line) AND (trial* or RCT* or random*). The reference lists of retrieved articles were also checked to identify any additional eligible publications.

The inclusion criteria were studies (1) that reported randomized controlled trials (RCTs) or cluster RCTs of internet-based interventions for ≥2 SNAP behaviors as either the sole intervention or an adjunct to written materials, (2) with adults aged 18 years or older, (3) that reported outcomes for ≥2 SNAP behaviors at least 6 months postrecruitment, (4) that had a no-intervention control group or the control group received information either in hard copy or information unrelated to SNAP via a website, and (5) in the English language in a peer-reviewed journal.

Publications were excluded if (1) they did not report the outcomes of an RCT or cluster RCT (eg, systematic reviews, commentaries); (2) they examined only 1 SNAP health behavior; (3) they included special populations only such as people with chronic conditions (eg, cancer, diabetes) or pregnant women (this criterion was chosen because people with chronic conditions may differ in their motivation and capacity to change behaviors compared with those without chronic conditions and is consistent with the criterion set in another systematic review that examined multiple health risk behaviors [16]); (4) were conducted with people under 18 years of age; (5) outcome measures were not related to SNAP (eg, blood pressure); (6) the internet-based intervention was part of a multicomponent approach that included other modes of support (eg, face-to-face, telephone); (7) there was no control arm, and instead, comparisons were made with other interventions (eg, face-to-face support); and (8) SNAP outcomes were measured before 6 months postrecruitment.

This systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [20]. All records identified in each electronic database were imported into Endnote, and duplicates were removed. Titles, abstracts, and full texts of each reference were independently screened in duplicate by 2 reviewers (LW and FT or AM or EB) to determine if eligibility criteria were met. Full-text articles were retrieved when eligibility could not be determined from the title and abstract screening.

Data were independently extracted from the included studies by 2 authors (AM and AH or LW). A third author (FT) resolved any inconsistencies in data extraction. The study and sample characteristics extracted from eligible publications included authors and year of publication, country, years that data were collected, setting, sample characteristics (eg, mean age, gender, education, employment status), recruitment method, eligibility criteria, treatment conditions (relevant arms only; ie, internet-based intervention and control arms), internet-based intervention received (eg, duration, number of modules), retention rate at follow-up, SNAP measures, SNAP outcomes at 6 months of follow-up or later, and costs.

The outcomes extracted for each health behavior were any measure of (1) tobacco smoking (eg, current tobacco smoking, point prevalence abstinence, or prolonged abstinence), (2) nutrition (eg, number of daily serves of fruit and vegetables, dietary score), (3) alcohol consumption (eg, number of alcoholic drinks per day), and (4) physical activity (eg, moderate to vigorous physical activity, metabolic equivalent of task [MET] minutes per week).

The Quality Assessment Tool for Quantitative Studies developed by the Effective Public Health Practice Project was used to assess methodological quality [21]. This tool was chosen because it has demonstrated content validity, construct validity, and test-retest reliability [22] and has been shown to have higher interrater reliability than the Cochrane Collaboration Risk of Bias Tool [23]. The Quality Assessment Tool for Quantitative Studies allows randomized trials to be rated on 6 components: (1) selection bias, (2) study design, (3) confounders, (4) blinding, (5) data collection methods, and (6) withdrawals and dropouts. Each study was rated as “strong,” “moderate,” or “weak” for each component. An overall global rating was then assigned to each study, with studies classified as “strong” (no weak ratings), “moderate” (1 weak rating), or “weak” (2 or more weak ratings). The Quality Assessment Tool for Quantitative Studies Dictionary was used to make judgments about each of the 6 components [24].

The methodological quality of included studies was rated by 2 authors (FT and AM). Any discrepancies were discussed between these authors until consensus was reached. When rating the data collection methods, the measures for “all” SNAP outcomes needed to be shown to be valid and reliable for the data collection methods to be rated as strong. For instance, if smoking cessation was measured via biochemical validation and physical activity assessed via pedometers, the data collection methods were rated as strong. However, if smoking cessation was measured via a self-reported measure with no information about its psychometric properties and pedometers were used to assess physical activity, the data collection methods were rated as weak because “all” SNAP measures were not shown to be reliable and valid.

A robust variance estimation meta-analysis was performed using the R package robumeta. All SNAP behaviors were coded so they were in the same direction, with higher scores equating to worse health risk outcomes. Where a study measured the outcome at multiple time points (eg, 6 months and 12 months), data from the longer-term follow-up was included in the meta-analysis. The SNAP outcomes from each study were converted into Cohen d (standardized mean differences [SMDs]) and the corresponding variance [25]. Robust variance estimation meta-analysis was then performed on the SMDs (and variances), using the R package robumeta. A common within-study correlation (rho) of 0.8 was assumed, and sensitivity analyses were performed to determine the sensitivity of the results to this assumption (by looking at the results across various rho equal to 0, 0.2, 0.4, 0.6, or 1.0). Heterogeneity was measured using the I² statistic [26].

The PRISMA flow diagram outlining the study selection process is presented in Figure 1. A total of 34,673 records were identified from the electronic database searches. After the removal of duplicates, 16,420 records had their title and/or abstract screened, and 16,341 did not meet the inclusion criteria. The full texts of the remaining 79 articles were retrieved for further review, and 68 articles were excluded for the following reasons: multicomponent intervention (n=29), not an RCT (n=19), did not measure 2 or more SNAP behaviors (n=7), less than 6 months of follow-up (n=6), did not include a no-intervention or self-help control group (n=4), not an internet-based intervention (n=2), and population with chronic disease (n=1). The inclusion criteria for this review were met by 11 studies.

The robust variance estimation meta-analysis found that, compared with the control group, internet-based interventions achieved an overall significant improvement across all SNAP behaviors in the long-term (SMD –0.12 [improvement as higher scores = worse health risk outcomes], 95% CI –0.19 to –0.05; I²=1.5%, P=.01) [27,29,30,32,34-37]. Heterogeneity was low (I²=1.5%), and the tau-square (the extent of variation due to between-study variance) was low, at <0.001. The sensitivity analysis showed that the model results did not vary greatly across different values of within-study correlation (rho).

Table 1 presents the methodological quality ratings for the 6 components and the global rating as assessed via the Quality Assessment Tool for Quantitative Studies [21]. In terms of selection bias, all studies were rated “weak,” as 9 of these studies recruited volunteers who responded to advertisements or sign-up tables [27-34,37], and 2 studies sent emails to all incoming undergraduate students but recruited less than half of those approached [35,36]. All included studies were RCTs or cluster RCTs and were rated as “strong” in terms of study design [27-37]. With regards to confounders, 8 studies were rated as “strong” because there were no between-group differences at baseline [31,32,34,35,37] or adjustments for baseline characteristics were made during analysis [27,29,36], while 1 study was rated as “moderate,” as stratification attempted to balance baseline characteristics across groups [33], and 2 studies were rated as “weak,” as no adjustments were made during analysis for baseline characteristics that differed between the groups [28,30]. For blinding, 9 studies were rated as “weak” because the assessors and participants were not blinded [28,31,34] or there was no information about blinding [27,29,30,33,35,36], while 2 studies were rated as “moderate” because only the assessors were blinded to the condition [32,37]. For data collection methods, 8 studies were rated as “weak,” as all relevant SNAP measures were not shown to be valid or reliable (either via the use of an objective measure or demonstration of acceptable psychometric properties of a self-reported measure) [27,30,31,33-37], while 3 studies were classified as “strong,” as all SNAP measures used were valid and reliable [28,29,32]. In relation to withdrawals and dropouts, 4 studies were rated as “strong,” with retention rates ≥80% (105/121; 935/1071; 367/423; 95/100) [27,33,34,37]; 4 studies were rated as “moderate,” with retention rates between 60% and 79% (422/606; 1126/1689; 309/441; 913/1445) [29,30,32,36]; and 3 studies were rated as “weak,” with retention rates <60% (148/317; 131/221; 1079/2621) [28,31,35]. In terms of the global rating, 1 study was rated as “moderate” [32], and the remaining 10 studies were rated as “weak” [27-31,33-37].

This is the first systematic review to examine the long-term effectiveness of internet-based interventions on SNAP behaviors collectively in adults aged 18 years or older. This systematic review focused on internet-based interventions to increase the homogeneity of included studies. This is similar to other systematic reviews that have focused on a specific digital technology [38,39]. More broadly, however, digital technologies can also include text messaging, email, mobile applications, video conferencing [40], and just-in-time feedback interventions [41]. The studies included in this systematic review most commonly examined effectiveness on 2 SNAP behaviors, namely nutrition and physical activity [27-33]. Only 3 studies examined the effectiveness of internet-based interventions on all 4 SNAP behaviors [35-37], whereas 1 study measured the effect on 3 behaviors (ie, tobacco smoking, nutrition, physical activity) [34].

The robust variance estimation meta-analysis findings reported that internet-based interventions achieved an overall significant improvement across all SNAP behaviors in the long term. This suggests that internet-based interventions that adopt a holistic approach to behavior change by addressing multiple SNAP behaviors improve these behaviors collectively and consequently may lead to better health outcomes and reduced health care costs. Given no previous systematic reviews have examined the long-term effectiveness of internet-based interventions on multiple SNAP behaviors nor provided an overall effect size across all SNAP behaviors, we cannot compare our findings to previous reviews. To advance the field, further research is needed on the long-term effectiveness of internet-based interventions on multiple SNAP behaviors.

The studies in this systematic review recruited participants from a variety of settings, including universities [29,30,34-36], the community [28,32,33,37], and health care [27,31]. Among the 5 studies conducted in universities, only 1 study reported a significant treatment effect on both nutrition and physical activity [30], while another study found the intervention reduced current smoking but had no effect on nutrition, alcohol, and physical activity [36]. Of 4 studies conducted in the community, 2 studies reported a significant treatment effect of the internet-based intervention on nutrition and some physical activity outcomes [32] or on some nutrition outcomes but not physical activity [33]. The trials that recruited participants from health care settings found no significant differences between the internet-based intervention and the control condition [27,31]. Additional research is needed across a variety of settings to expand the evidence base examining the long-term effectiveness of internet-based interventions on combinations of 2 or more SNAP behaviors.

In terms of methodological quality, 10 of 11 studies had a global rating of “weak” [27-31,33-37], with only 1 study rated as “moderate” [32]. Improvements to methodological rigor are particularly needed for selection bias, blinding, and data collection methods. Selection bias could be reduced by using recruitment methods that aim to enroll a representative sample (eg, random selection of potential participants) while data collection methods could be improved via objective measures (eg, pedometers for physical activity, biochemical validation for smoking cessation) for all SNAP behaviors assessed. Given the nature of behavioral interventions, blinding is often difficult; however, future studies should attempt to blind assessors and participants where possible.

This systematic review had some limitations. First, although we were able to pool the studies to undertake a robust variance estimation meta-analysis to examine the long-term effectiveness of internet-based interventions across all SNAP behaviors, additional analyses examining potential moderators (eg, country) were not possible due to the relatively small number of studies in this systematic review. Second, the methodological quality assessments were based on the information contained in the articles, and missing details from these articles may have had an impact on the ratings. Finally, all the studies were conducted in high-income countries, which may limit the generalizability of this systematic review’s findings to low- and middle-income countries. In addition to expanding the research in the settings and populations included in this review, future research should assess the long-term effectiveness of internet-based interventions on multiple SNAP behaviors in additional populations (eg, culturally and linguistically diverse groups, Indigenous), settings (eg, vocational education settings, rural and remote locations), and countries (eg, low- and middle-income) to strengthen the evidence base and improve the generalizability of the findings.

Internet-based interventions were found to produce an overall significant improvement across all SNAP behaviors in the long term. Given the promising findings on the long-term effectiveness of internet-based interventions across all SNAP behaviors collectively, such interventions may maximize improvements to health and prevent chronic diseases.