This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The Internet is increasingly used as a medium for the delivery of interventions designed to promote health behavior change. However, reviews of these interventions to date have not systematically identified intervention characteristics and linked these to effectiveness.
The present review sought to capitalize on recently published coding frames for assessing use of theory and behavior change techniques to investigate which characteristics of Internet-based interventions best promote health behavior change. In addition, we wanted to develop a novel coding scheme for assessing mode of delivery in Internet-based interventions and also to link different modes to effect sizes.
We conducted a computerized search of the databases indexed by ISI Web of Knowledge (including BIOSIS Previews and Medline) between 2000 and 2008. Studies were included if (1) the primary components of the intervention were delivered via the Internet, (2) participants were randomly assigned to conditions, and (3) a measure of behavior related to health was taken after the intervention.
We found 85 studies that satisfied the inclusion criteria, providing a total sample size of 43,236 participants. On average, interventions had a statistically small but significant effect on health-related behavior (d+ = 0.16, 95% CI 0.09 to 0.23). More extensive use of theory was associated with increases in effect size (
The review provides a framework for the development of a science of Internet-based interventions, and our findings provide a rationale for investing in more intensive theory-based interventions that incorporate multiple behavior change techniques and modes of delivery.
In June 2009 an estimated 25% of the world’s population had access to the Internet, with estimates in Europe and North America being considerably higher (50% and 74%, respectively) [
Quantitative reviews of Internet-based interventions report positive–albeit highly variable and often small–effects on behaviors such as physical activity, tobacco use, exercise, and so on [
Three intervention characteristics may influence the impact on behavior [
Theoretical basis refers to the theory or theories used to develop the intervention. For example, in an effort to promote physical activity, Spittaels et al [
Behavior change techniques refer to the specific strategies used in the intervention to promote behavior change. For example, some interventions designed to promote smoking abstinence prompt barrier identification and problem solving (eg, [
The interventions in the present review were delivered via the Internet. The effects of this primary mode of delivery can be estimated by examining studies that compare similar materials presented via the Internet versus other modes, such as print [
The present review sought to investigate which characteristics of Internet-based interventions were associated with effectiveness. By so doing, we answer the important applied and theoretical questions: Which theories should researchers draw on in developing interventions? How can theory best be used to inform Internet-based interventions? What behavior change techniques are effective when employed via the Internet? Is the mode by which the intervention is delivered important?
In July 2008 we conducted a computerized search using ISI Web of Knowledge, which covers a number of databases including Web of Science conference proceedings (1900-), BIOSIS Previews (1985-), and Medline (1950-). We used the following search terms: Web-based, Internet, digital, online, technolog*, computer, treatment, RCT, trial, intervention, behavio* change. (The asterisk automatically truncates the term such that, for example, technolog* will also find technology, technologies, etc). Studies had to include one or more of the search terms in the title. We also sent an email to the distribution list of the European Health Psychology Society to request unpublished research. There were three inclusion criteria for the meta-analysis. First, the primary components of the intervention must have been delivered via the Internet (not including CD-ROMs, SMS messaging, or other computer applications). Second, the studies must have involved random assignment of participants to a treatment group that received an Internet-based intervention and a comparison group that received either a control intervention or no intervention. Finally, a measure of behavior related to health must have been taken after the intervention. We did not include studies that only measured symptoms (eg, anxiety, depression), cognitions (eg, attitudes, intentions), outcomes presumed to be the consequence of behavioral changes (eg, weight loss, blood glucose levels), or behaviors unrelated to health (eg, use of literature services).
Flow of information through the different phases of the review (adapted from [
The effect size for post-intervention behavior differences between the conditions was calculated in line with Cochrane recommendations [
The coding scheme developed by Michie and Prestwich [
In addition to considering each use of theory separately, we also summed items 1 through 11 to create an overall “use of theory” score that could be used to evaluate whether more extensive use of theory leads to larger effects than less extensive use. In a slight change to the published recommendations, item 8 (“At least one, but not all, intervention techniques are explicitly linked to at least one theory-relevant construct/predictor”) was coded as “yes” if item 7 (“All intervention techniques are explicitly linked to at least one theory-relevant construct/predictor”) was coded as “yes.” Similarly, item 11 (“At least one, but not all, theory-relevant constructs/predictors are explicitly linked to at least one intervention technique”) was coded as “yes” if item 10 (“All theory-relevant constructs/predictors are explicitly linked to at least one intervention technique”) was coded as “yes.” This ensured that when we created the ”use of theory” score, reports that linked, for example, all theoretical constructs with intervention techniques, were also credited as linking some theoretical constructs with intervention techniques.
Interventions were coded as having a particular theoretical basis only if the theory was used to develop the intervention techniques (item 5 of the coding scheme of Michie and Prestwich [
The behavior change techniques used in the interventions were coded using an augmented 40-item version [
Mode of delivery was coded using a novel coding scheme developed by the present authors. For convenience, we divided mode of delivery into (i) automated functions, (ii) communicative functions, and (iii) use of supplementary modes. Each category included a list of delivery modes, and we marked whether or not each intervention used that mode. Automated functions included: (a) the use of an enriched information environment (eg, supplementary content and links, testimonials, videos, or games), (b) automated tailored feedback based on individual progress monitoring (eg, comparison to norms or goals, reinforcing messages, or coping messages), and (c) automated follow-up messages (eg, reminders, tips, newsletters, encouragement). Communicative functions included: (d) access to an advisor to request advice (eg, “ask the expert” facility, expert-led discussion board, or chat sessions), (e) scheduled contact with advisor (eg, emails), and (f) peer-to-peer access (eg, buddy systems, peer-to-peer discussions boards, forums, or live chat). Finally, use of supplementary modes included the use of (g) email, (h) telephone, (i) Short Messaging Service (SMS), (j) CD-ROM, or (k) videoconferencing.
The features of intervention delivery that we coded were, to a large extent, constrained by the features that authors typically report and that can be easily and objectively verified (eg, whether text messages were used). The list is not intended to be exhaustive and we recognize that there are other features that may be important but that are not routinely used or reported, or that are hard to measure. For example, navigational format (eg, the extent to which users are “tunnelled” to particular information vs given free choice [
We used Hedges
The weighted average effect size across all interventions was d+= 0.16 with a 95% confidence interval from 0.09 to 0.23 based on 85 studies (k = 85) and a total of 43,236 participants (see
We also calculated effect sizes separately for commonly targeted behaviors (see
Weighted effect sizes (d+) for behavior change as a function of Internet-based interventions by behavior type
Behavior |
|
|
95% CI | d+ c |
Physical activity | 20 | 128.76f | 0.09-0.38 | 0.24e |
Dietary behavior | 10 | 30.82e | 0.02-0.37 | 0.20f |
Alcohol consumption | 9 | 47.45f | 0.00-0.27 | 0.14d |
Smoking abstinence | 12 | 45.46e | -0.04 to 0.18 | 0.07 |
Interventions targeting multiple behaviors | 10 | 7.90 | 0.08-0.17 | 0.12f |
Interventions targeting a single behavior | 75 | 879.81f | 0.09-0.24 | 0.17f |
All studies | 85 | 896.67f | 0.09-0.23 | 0.16f |
ak = the number of interventions included in the estimate of effect size
bQ = homogeneity for the subgroup of interventions
cd+ = weighted average effect size
d
e
f
Across all interventions, the homogeneity Q statistic was highly significant (Q = 896.67,
Of the different uses of theory proposed by Michie and Prestwich’s coding scheme [
Only three theories were used by three or more studies to develop the intervention; social cognitive theory (SCT) [
The most commonly used behavior change techniques (used by 30% or more of interventions) were providing information on the consequences of behavior in general (k = 29), prompting self-monitoring of behavior (k = 28), and identifying barriers and/or problem solving (k = 26). The largest effects on behavior were observed for interventions that provided stress management (d+= 0.50, 95% CI 0.27 to 0.72) or general communication skills training (d+= 0.49, 95% CI 0.25 to 0.73), although these were used by relatively few interventions (k = 5 and 3, respectively). Modeling, relapse prevention/coping planning, facilitating social comparison, goal setting, action planning, and provision of feedback on performance all had effects on behavior that exceeded d+= 0.20 (Median d+= 0.28). Finally, a few strategies had small and non-significant effects on behavior: use of follow-up prompts, self-monitoring of behavioral outcome, emotional control training, and provision of information about others approval. Overall, meta-regression indicated that the number of behavior change techniques employed had a significant positive impact on effect size (ß = 0.36,
Only one mode of delivery was used by 30% or more of interventions–providing an enriched information environment (k = 30). Over 20% of interventions, however, provided access to an advisor to request advice (k = 23), used peer-to-peer access (k = 20), used email in addition to the Internet-based intervention (k = 19), or provided automated tailored feedback (k = 18). For convenience of interpretation, effect sizes for modes of delivery were divided into three subgroups: automated functions, communicative functions, and use of supplementary modes. In terms of automated functions, small, but significant, effects on behavior were observed for interventions that provided automated tailored feedback (d+= 0.18, k = 18, 95% CI 0.07 to 0.28) or an enriched information environment (d+= 0.15, k = 30, 95% CI 0.07 to 0.23). Interventions that provided automated follow-up messages tended not to have significant effects on behavior (d+= 0.09, k = 14, 95% CI -0.01 to 0.19). Of the communicative functions, interventions that provided access to an advisor to request advice tended to have small-to-medium effects on behavior (d+= 0.29, k = 23, 95% CI 0.16 to 0.42), while smaller effects on behavior were observed for interventions that provided scheduled contact with an advisor (d+= 0.22, k = 13, 95% CI 0.09 to 0.36) or peer-to-peer access (d+= 0.20, k = 20, 95% CI 0.09 to 0.21). Finally, use of additional modes appeared to have distinct effects on behavior change with Internet-based interventions that also used text messages having large effects on behavior (d+= 0.81, k = 4, 95% CI 0.14 to 1.49), Internet-based interventions using the telephone having small-to-medium effects (d+= 0.35, k = 7, 95% CI 0.09 to 0.61), and interventions using email as an additional mode of delivery tending to have small effects on behavior (d+= 0.18, k = 19, 95% CI 0.07 to 0.29).
Effect sizesa by theoretical basis, use of theory, behavior change techniques, and mode of delivery. The numbering for use of theory, behaviour change techniques, and the letters for mode of delivery correspond with those items in the coding frames and
Kb | Qc | 95% CI | d+ d | |||
|
||||||
Theory of reasoned action/planned behavior (TPB) [ |
9 | 108.44h | 0.15 to 0.56 | 0.36g | ||
Transtheoretical model (TTM) [ |
12 | 68.99h | 0.08 to 0.33 | 0.20g | ||
Social cognitive theory (SCT) [ |
12 | 18.62 | 0.04 to 0.25 | 0.15g | ||
Elaboration likelihood model (ELM) [ |
2 | |||||
Extended parallel process model (EPPM) [ |
1 | |||||
Self-regulation theory (SRT) [ |
1 | |||||
Precaution adoption process model (PAPM) [ |
1 | |||||
Diffusion of innovations model (DIM) [ |
1 | |||||
Health belief model (HBM) [ |
1 | |||||
Social norms theory (SNT) [ |
1 | |||||
|
||||||
4. Theory/predictors used to select recipients for the intervention | 3 | 2.84 | 0.15 to 0.52 | 0.33h | ||
9. Group of techniques are linked to a group of constructs/predictors | 6 | 9.85 | 0.03 to 0.43 | 0.23f | ||
5. Theory/predictors used to select/develop intervention techniques | 37 | 191.40h | 0.13 to 0.29 | 0.21h | ||
2. Targeted construct mentioned as predictor of behavior | 18 | 60.07h | 0.11 to 0.31 | 0.21g | ||
6. Theory/predictors used to tailor intervention techniques to recipients | 11 | 67.75h | 0.07 to 0.34 | 0.21g | ||
1. Theory/model of behavior mentioned | 30 | 161.33h | 0.11 to 0.28 | 0.19h | ||
8. At least one of the intervention techniques is linked to theory | 19 | 93.65h | 0.09 to 0.29 | 0.19g | ||
3. Intervention based on single theory | 12 | 57.13h | 0.05 to 0.32 | 0.18f | ||
10. All theory-relevant constructs are linked to intervention techniques | 10 | 47.70h | -0.02 to 0.37 | 0.18 | ||
11. At least one of the theory-relevant constructs is linked to an intervention |
18 | 70.63h | 0.07 to 0.27 | 0.17g | ||
7. All intervention techniques are linked to theory | 2 | |||||
|
||||||
35. Stress management | 5 | 6.73 | 0.27 to 0.72 | 0.50h | ||
39. General communication skills training | 3 | 4.38 | 0.25 to 0.73 | 0.49h | ||
21. Model/demonstrate the behavior | 5 | 24.80h | -0.01 to 0.70 | 0.35e | ||
34. Relapse prevention/coping planning | 14 | 38.31h | 0.17 to 0.47 | 0.32h | ||
27. Facilitate social comparison | 4 | 3.25 | 0.04 to 0.55 | 0.29f | ||
5. Goal setting (behavior) | 25 | 126.24h | 0.16 to 0.38 | 0.27h | ||
7. Action planning | 18 | 101.67h | 0.13 to 0.37 | 0.25h | ||
19. Provide feedback on performance | 19 | 77.38h | 0.09 to 0.34 | 0.22g | ||
8. Barrier identification/problem solving | 26 | 112.52h | 0.10 to 0.30 | 0.20h | ||
20. Provide instruction | 25 | 97.95h | 0.13 to 0.28 | 0.20h | ||
22. Teach to use prompts/cues | 3 | 5.45 | -0.17 to 0.57 | 0.20 | ||
4. Provide normative information about others’ behavior | 16 | 94.32h | 0.07 to 0.28 | 0.18g | ||
28. Plan social support/social change | 15 | 41.32h | 0.10 to 0.27 | 0.18h | ||
13. Provide rewards for behavior | 7 | 7.17 | 0.09 to 0.28 | 0.18h | ||
16. Prompt self-monitoring of behavior | 28 | 80.81h | 0.07 to 0.24 | 0.16h | ||
1. Provide information on the consequences in general | 29 | 114.14h | 0.06 to 0.21 | 0.14h | ||
2. Provide information on the consequences for individual | 12 | 47.57h | 0.04 to 0.24 | 0.14g | ||
26. Use of follow up prompts | 5 | 39.35h | -0.10 to 0.35 | 0.13 | ||
17. Prompt self-monitoring of behavioral outcome | 13 | 45.73h | -0.03 to 0.26 | 0.12 | ||
12. Reinforcing effort toward behavior | 3 | 2.89 | 0.02 to 0.19 | 0.11f | ||
36. Emotional control training | 11 | 35.39h | -0.03 to 0.22 | 0.09 | ||
3. Provide information about others’ approval | 5 | 10.48f | -0.11 to 0.23 | 0.06 | ||
6. Goal setting (outcome) | 2 | |||||
10. Prompt review of behavioral goals | 2 | |||||
14. Shaping | 2 | |||||
23. Environmental restructuring | 2 | |||||
25. Prompt practice | 2 | |||||
24. Agree behavioral contract | 1 | |||||
31. Fear Arousal | 1 | |||||
32. Prompt self-talk | 1 | |||||
37. Motivational interviewing | 1 | |||||
9. Set graded tasks | 0 | |||||
11. Prompt review of outcome goals | 0 | |||||
15. Prompting generalisation of behavior | 0 | |||||
18. Prompting focus on past success | 0 | |||||
29. Prompt identification as role model | 0 | |||||
30. Prompt anticipated regret | 0 | |||||
33. Prompt use of imagery | ||||||
38. Time management | ||||||
40. Provide non-specific social support | ||||||
|
||||||
b. Automated tailored feedback | 18 | 83.75h | 0.07 to 0.28 | 0.18g | ||
a. Enriched information environment | 30 | 117.24h | 0.07 to 0.23 | 0.15h | ||
c. Automated follow-up messages | 14 | 49.81h | -0.01 to 0.19 | 0.09 | ||
|
||||||
d. Access to advisor to request advice | 23 | 121.15h | 0.16 to 0.42 | 0.29h | ||
e. Scheduled contact with advisor | 13 | 35.70h | 0.09 to 0.36 | 0.22g | ||
f. Peer-to-peer access | 20 | 88.21h | 0.09 to 0.21 | 0.20h | ||
|
||||||
i. Text message (SMS) | 4 | 39.22h | 0.14 to 1.49 | 0.81a | ||
h. Telephone | 7 | 19.02g | 0.09 to 0.61 | 0.35g | ||
g. Email | 19 | 143.98h | 0.07 to 0.29 | 0.18g | ||
j. CD-ROM | 1 | |||||
k. Videoconferencing | 1 |
aEffect sizes are ordered within category by size of effect. Characteristics supported by less than three interventions were not examined in order to ensure reliable evaluations of the impact of particular intervention characteristics on effect size.
bk = the number of interventions included in the estimate of effect size
cQ = homogeneity across the subgroup of interventions
dd+ = weighted average effect size
eRemoving Mikolajczak et al [
f
g
h
The primary aim of the present review was to relate the characteristics of Internet-based interventions to their effectiveness in promoting health behavior change. Like previous reviews, the interventions tended to have variable effects on behavior (ie, the homogeneity Q statistic was significant), and the average effect on behavior was statistically small. Thus, while some interventions had very large effects (d > 1.00) on behavior (eg, [
Interventions differed substantially in their use of theory, but more extensive use of theory was associated with larger effect sizes. This finding is consistent with assertions that interventions can benefit from using behavior change theory [
The finding that interventions that incorporated more behavior change techniques tended to have larger effects than interventions that incorporated fewer techniques justified the investment in relatively elaborate interventions. This finding may be a consequence of different techniques targeting different aspects of the behavior change process [
The two behavior change techniques that were associated with the greatest changes in behavior were stress management and general communication skills training. It is interesting that both techniques influence behavior change indirectly via mechanisms such as facilitating problem-solving, promoting self-efficacy [
Two other findings in relation to behavior change techniques warrant comment. First, it was notable that providing information about others’ approval (subjective or injunctive norms) seemed to be less effective than providing normative information about others’ behavior (descriptive norms, d+= 0.06 and 0.18, respectively). This finding supports the distinction between the two types of normative influence [
The present review developed a novel coding scheme for the mode by which Internet-based interventions are delivered. Dividing mode of delivery into automated functions, communicative functions, and use of supplementary modes proved informative, with distinct effects being identified within each category. Text messages were highly effective and used in several ways: to promote interaction with the intervention [
The present review is, to our knowledge, the first to systematically code the characteristics of Internet-based interventions designed to promote behavior change and to link these characteristics to effect size. The strengths of the review are the systematic, meta-analytic approach, the use of established coding frames where possible, and the large number of different interventions that focus on a range of different behaviors. The findings suggest that the effectiveness of Internet-based interventions is associated with more extensive use of theory (in particular the use of the theory of planned behavior), inclusion of more behavior change techniques, and use of additional methods of interacting with participants (especially text messages). The review provides a framework for research that can contribute to a science of Internet-based interventions [
This review was inspired by a workshop on Internet-based behaviour change interventions in addiction sponsored by the Society for the Study of Addiction. The authors would like to thank Hongmei Han, Marney White, and Donald Williamson for providing additional information concerning their research. We also thank Craig Whittington for statistical assistance and Robert West for helpful comments on earlier drafts of this manuscript. This review was funded in part by an ESRC grant (RES-149-25-1069) awarded to LY and SM. This grant funds the Southampton “LifeGuide” node of the National Centre for e-Social Science (www.lifeguideonline.org) and supported JJ.
Effect Sizes for Interventions Included in the Meta-Analysis
Intervention Characteristics for Interventions Included in the Meta-Analysis
diffusion of innovations model
elaboration likelihood model
extended parallel process model
health belief model
precaution adoption process model
social cognitive theory
short message service
social norms theory
self-regulation theory
theory of reasoned action/planned behavior
transtheoretical model