This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Researchers and practitioners have developed numerous online interventions that encourage people to reduce their drinking, increase their exercise, and better manage their weight. Motivations to develop eHealth interventions may be driven by the Internet’s reach, interactivity, cost-effectiveness, and studies that show online interventions work. However, when designing online interventions suitable for public campaigns, there are few evidence-based guidelines, taxonomies are difficult to apply, many studies lack impact data, and prior meta-analyses are not applicable to large-scale public campaigns targeting voluntary behavioral change.
This meta-analysis assessed online intervention design features in order to inform the development of online campaigns, such as those employed by social marketers, that seek to encourage voluntary health behavior change. A further objective was to increase understanding of the relationships between intervention adherence, study adherence, and behavioral outcomes.
Drawing on systematic review methods, a combination of 84 query terms were used in 5 bibliographic databases with additional gray literature searches. This resulted in 1271 abstracts and papers; 31 met the inclusion criteria. In total, 29 papers describing 30 interventions were included in the primary meta-analysis, with the 2 additional studies qualifying for the adherence analysis. Using a random effects model, the first analysis estimated the overall effect size, including groupings by control conditions and time factors. The second analysis assessed the impacts of psychological design features that were coded with taxonomies from evidence-based behavioral medicine, persuasive technology, and other behavioral influence fields. These separate systems were integrated into a coding framework model called the communication-based influence components model. Finally, the third analysis assessed the relationships between intervention adherence and behavioral outcomes.
The overall impact of online interventions across all studies was small but statistically significant (standardized mean difference effect size d = 0.19, 95% confidence interval [CI] = 0.11 - 0.28,
These findings demonstrate that online interventions have the capacity to influence voluntary behaviors, such as those routinely targeted by social marketing campaigns. Given the high reach and low cost of online technologies, the stage may be set for increased public health campaigns that blend interpersonal online systems with mass-media outreach. Such a combination of approaches could help individuals achieve personal goals that, at an individual level, help citizens improve the quality of their lives and at a state level, contribute to healthier societies.
Research suggests that online intervention can motivate people to adopt healthy behaviors, such as reducing binge drinking [
At present, numerous factors are driving health promotion campaigns online. First, the Internet offers health campaigners a convenient channel to increase the reach of large-scale campaigns. The Internet is now a major source of information for health advice [
Fourth, the cost-effectiveness of preventative medicine and online outreach are both driving the innovation of online health solutions. Governments are recognizing that it is more cost-effective to market healthy lifestyles rather than pay to treat the outcomes of unhealthy lifestyles [
When designing campaigns to enhance citizen well-being, health officials draw from numerous fields, theories, frameworks, and techniques. With almost 40 years of academic and practical development, social marketing is an established approach to behavioral change [
Social marketers frequently use the Internet to promote healthy lifestyles as part of multichannel campaigns, increasingly with social media tools. However, several authors have argued that new media have introduced changes that are shifting how social marketing campaigns should be carried out and that the old one-way communication model does not make sense in online environments [
To understand how online intervention design can influence users' behaviors, some researchers have examined health behavioral change interventions that can be found through Internet search engines. Their studies tend to offer uncertain and sometimes pessimistic conclusions. One evaluation of existing health behavioral change websites concluded that many of these sites did not include the basic requirements to achieve health behavior change [
Other types of research that can inform intervention design include meta-analyses [
Thus far, no meta-analyses have quantified how the psychological design of online interventions can influence behaviors that are typically targeted in social marketing campaigns. To overcome this gap, there is a need to identify a sample of online behavioral change interventions that resembles those used in large-scale public health campaigns and which also offers insight into the psychological architectures associated with voluntary behavioral change.
In clinical studies, the more people adhere to lifestyle change programs, the more their health improves. Similarly, those with life threatening diseases who stick to diet and lifestyle programs can potentially prevent their condition from worsening [
Research suggests that exposure to programs (their dose), is a key predictor of behavior change. In one systematic review, the majority of participants failed to engage in more than half of the expected eHealth activities. However, those interventions with high utilization showed better behavioral outcomes [
In this paper, the term
There are two types of adherence. First,
Although online interventions are frequently described as a homogenous group, they may be radically different in terms of their purpose, design, and psychological architectures. In order to describe the diversity of existing online interventions, any coding system would need to accommodate a large variety of complex factors that may explain intervention efficacy. However, there is no consensus on what constitutes the best theoretical framework or list of factors that may be used to describe interventions and which may also explain their efficacy. The literature offers numerous competing behavioral change theories and taxonomies that are founded on different assumptions, application contexts, and academic disciplines. This has resulted in numerous overlapping and ill-fitting taxonomies, none of which is comprehensive enough to describe online interventions on their own [
To overcome the lack of intervention design guidelines addressing behavioral outcomes, this study first reviewed numerous influence systems and then developed a communication-based framework to consolidate taxonomies across various fields into a simple coding system. When describing these various systems, the following terms are used:
In order to develop a comprehensive coding system to describe the psychological architectures of online interventions, a model was developed to consolidate influence systems across a range of fields. It is called the communication-based influence components model (CBICM). The model views interaction between audiences and online interventions as roughly equal to the relationship between a therapist and client, where the therapist’s treatment is just one of many factors that may explain efficacy. For instance, many therapists may offer the same treatment to their patients; however, for some therapists, their reputation, communication style, flexibility, and willingness to adapt to the client’s needs can influence the efficacy of their treatment. The CBICM is based on the principle that the strength of an intervention is the result of its influence components [
This meta-analysis assessed online intervention features that can be used to guide the development of population-wide campaigns targeting voluntary lifestyle behaviors. Furthermore, it assessed relationships proposed under the law of attrition, which offers insights into the role of intervention exposure (dose) and intervention efficacy. Toward these objectives, the study assesses psychological design factors, time trends, and the role of dose in online interventions.
To identify qualifying studies for this meta-analysis, a 3-step systematic review approach was used [
Second, these terms were used to identify and retrieve abstracts from relevant databases. In all, 5 bibliographic databases were selected. To cover the timeframe from 1999 through 2008, these databases were searched on September 20, 2008, and then on January 16, 2009, to cover 2008. The outcomes from both search sessions resulted in the following number of potential studies: 652 from Web of Knowledge, 292 from PsycINFO, 244 from MEDLINE, 327 from PubMed, and 7 from the Cochrane Library.
Third, additional strategies were employed to identify potential studies from the gray literature. A total of 59 additional studies were retrieved from the bibliographies of similar meta-analyses [
Eligible studies for this meta-analysis included published or unpublished research and reports in English. Qualifying papers included experimental, quasi-experimental, and correlational studies, including those with randomized and nonrandomized allocations. The substantive criteria in
Inclusion and exclusion criteria
Area | Criteria |
Timeframe | Inclusion: Years 1999 through 2008 |
Age | Inclusion: Preteens to older persons |
Behavioral domains | Inclusion: Health, safety, environmental, and community development behaviors |
Behavioral outcome (dependent variable) | Inclusion: A clear behavioral change outcome |
Intervention types | Inclusion: Web-based or Web and email-based |
Intervention mechanism | Inclusion: Primarily automated interventions (human-computer) |
Control treatments | Inclusion: Control group intervention comprising print, Web-based interventions, waitlists, placebos, and therapists |
Selection process flow chart
In total, 31 studies were included in this meta-analysis and coded. There were 2 studies that met the inclusion criteria that were removed from the overall analysis but were included in the dose analysis. The first study [
To evaluate the studies and test for potential publication bias, three validity assessment methods were employed: research quality assessment, cumulative meta-analysis, and a funnel plot assessment [
Second, a cumulative meta-analysis did not show that small studies were contributing a large impact on the final effect size. Thus, the small studies are unlikely to be biasing the sample of studies [
Third, the funnel plot in
Funnel plot of interventions
Publication bias is conventionally assessed according to three categories:
Data was extracted from studies using calculations by Borenstein et al and Lipsey and Wilson [
For the analysis of psychological design, the CBICM was used as a framework to group influence components from various influence systems. When coding influence components, 2 approaches were used. First,
For the dose analysis, when coding the adherence variables, study adherence was measured as the percentage of participants in a study at a given time compared with the baseline. Coding intervention adherence was more challenging, as it was conceived and reported in many ways. Across studies, intervention adherence was reported as log-ins, visits, page views, core pages viewed, percent of required reading completed, and complex multi-item measures. Researchers reported intervention adherence by the total number of users, averages per user, or percentages over various time units. In some cases, the variables were measured on continuous scales, in others, they were dichotomous, but more often, continuous variables were cut into arbitrary categories, such as high/low log-in groups. To deal with this diversity, 2 coding and meta-analytical approaches were employed to assess the relationship between intervention adherence and behavioral outcomes. The first approach coded any reported intervention adherence construct, while the second approach only coded adherence constructs that could be converted into a percentage.
Full intention to treat groups may distort the results by including many unmotivated participants, while the fully exposed group are likely to represent the most motivated participants [
This study presents three analyses. The first analysis provides the overall effect size estimates, including groupings by control conditions and time moderators. The second analysis assesses psychological design features, presenting overall correlations, descriptive statistics, and behavioral outcomes associated with influence components. The third analysis examines correlations between adherence variables and behavioral outcomes.
Following recommendations to select statistical models a priori on the basis of substantive justifications [
The majority of studies were randomized controlled trials, measured with continuous or dichotomous data with pre and post measures, while in some cases only post measures were reported. For group contrasts, that is, between-subject studies, the standardized mean difference, d, was used as the primary effect size measure. To assess categories used to explain heterogeneity in the analogue to ANOVA, the between-group heterogeneity statistic and its significance value Qb (
Interventions
Author (Year) and Reference Number | Experimental and Control Groups | Experimental Group | Research Score (%) | |||||
Pre (n) | Post (n) | Participant |
Mean Age | Male (%) | Study |
Intervention |
||
Bersamin et al (2007) [ |
139 | 139 | Students (who drink alcohol) | 18 | 48.0% | 57.4% | 73.1% | |
Bewick et al (2008) [ |
506 | 317 | Students | 21.3 | 31.0% | 59% | 73.1% | |
Bruning Brown et al (2004) a [ |
153 | 153 | Students (female) | 15.1 | 0.0% | 66.7% | 69.2% | |
Bruning Brown et al (2004) b [ |
69 | 69 | Parents | 3.4% | 100.0% | 50.0% | 69.2% | |
Celio et al (2000) [ |
52 | 47 | Students (female) | 19.6 | 0.0% | 96.3% | 71.0% | 92.3% |
Chiauzzi et al (2005) [ |
265 | 215 | Students (who are heavy drinkers) | 20 | 44.8% | 80.2% | 86.0% | 80.8% |
Dunton and Robertson (2008) [ |
155 | 128 | Women | 42.8 | 0.0% | 78.6% | 92.3% | |
Gueguen and Jacob (2001) [ |
1008 | 1008 | French citizens | 61.5% | ||||
Hunter et al (2008) [ |
451 | 446 | Military personnel | 33.5 | 50.0% | 85.0% | 80.8% | |
Jacobi et al (2007) [ |
97 | 97 | Students (female) | 22.5 | 0.0% | 100.0% | 83.0% | 80.8% |
Kim and Kang (2006) [ |
50 | 50 | Diabetics | 55.1 | 53.4% | 73.1% | ||
Kosma et al (2005) [ |
151 | 75 | Disabled persons | 45.5% | 84.6% | |||
Kypri et al (2004) [ |
104 | 83 | Students | 19.9 | 82.4% | 100.0% | 76.9% | |
Kypri and McAnally (2005) [ |
146 | 122 | Students | 20.3 | 46.0% | 82.0% | 100.0% | 76.9% |
Lenert et al (2004) [ |
485 | 144 | Smokers | 39 | 42.0% | 26.0% | 57.7% | |
Marshall et al (2003) [ |
655 | 258 | University faculty and staff | 43 | 50.0% | 76.5% | 26.0% | 73.1% |
McConnon et al (2007) [ |
221 | 131 | Obese persons | 45.8 | 23.0% | 48.7% | 53.0% | 76.9% |
McKay et al (2001) [ |
78 | 68 | Diabetics | 52.3 | 18.0% | 92.1% | 84.6% | |
Moore et al (2005) [ |
100 | 100 | Students | 21.7 | 42.2% | 86.2% | 65.4% | |
Napolitano et al (2003) [ |
65 | 52 | Hospital staff | 42.8 | 16.1% | 70.0% | 80.8% | |
Oenema et al (2005) [ |
521 | 384 | Employees | 42 | 57.0% | 72.0% | 69.2% | |
Petersen et al (2008) [ |
4254 | 4254 | Employees | 21.2% | 38.5% | |||
Roberto (2007) [ |
378 | 103 | Students (high school) | 15.5 | 41.7% | 84.8% | 88.5% | 53.8% |
Severson et al (2008) [ |
2523 | 1801 | Smokeless tobacco users | 36.7 | 97.9% | 44.1% | 50.0% | 57.7% |
Strecher et al (2005) [ |
3501 | 3501 | Smokers trying to quit with the nicotine patch | 36.9 | 43.5% | 46.6% | 80.8% | |
Strom et al (2000) [ |
102 | 45 | Headache sufferers | 41.5 | 25.0% | 39.2% | 80.8% | |
Swartz et al (2006) [ |
351 | 274 | Employees | 40.9 | 46.8% | 50.9% | 70.2% | 80.8% |
Tate et al (2001) [ |
91 | 81 | Overweight persons | 40.6 | 11.0% | 78.3% | 96.2% | |
Verheijden et al (2004) [ |
146 | 130 | Persons at risk of cardiovascular disease | 62 | 72.0% | 84.9% | 32.9% | 84.6% |
Winett et al (2007) [ |
707 | 620 | Church congregation | 53.13 | 33.0% | 88.5% | 57.0% | 57.7% |
Demographic descriptives
Demographic Descriptives | k | n | Percent | |
|
26 | 6028 | 100% | |
Men | 3152 | 52.3% | ||
Women | 2876 | 47.7% | ||
|
15 | 2341 | 100% | |
Bachelor’s level | 1347 | 57.6% | ||
Master’s level | 552 | 23.6% | ||
Secondary | 404 | 17.2% | ||
Primary | 38 | 1.6% | ||
|
19 | 2957 | 100% | |
White | 2475 | 83.7% | ||
African | 144 | 4.9% | ||
Mixed | 116 | 3.9% | ||
Asian | 82 | 2.8% | ||
Latin American | 74 | 2.5% | ||
Aboriginal | 33 | 1.1% | ||
Unclassified | 33 | 1.1% |
Effect size estimates
Groupings | k | d (95% confidence |
|
Qb ( |
Qw ( |
I2 | |
|
30 | N/A | |||||
All interventions | 30 | 0.194 (0.111 - 0.278) | < .001 | 64.125 (< .001) | 54.776 | ||
|
30 | 9.109 (.01) | |||||
Waitlist or placebo | 18 | 0.282 (0.170 - 0.393) | < .001 | 55.163 (< .001) | 69.183 | ||
Website | 8 | 0.162 (0.006 - 0.318) | .04 | 0.650 (.10) | < 0.001 | ||
4 | -0.110 (-0.343 to 0.123) | .35 | 1.623 (.65) | < 0.001 | |||
|
30 | 6.611 (.16) | |||||
Single-session | 4 | 0.404 (0.130 - 0.677) | .004 | 0.367 (.95) | < 0.001 | ||
From 2 days to 1 month | 5 | 0.205 (0.026 - 0.383) | .024 | 4.511 (.34) | 11.336 | ||
Over 1 month to 4 months | 16 | 0.220 (0.116 - 0.324) | < .001 | 30.131 (.01) | 50.218 | ||
Over 4 months to 7 months | 3 | 0.090 (-0.077 to 0.258) | .29 | 3.235 (.20) | 38.186 | ||
Over 7 months to 13 months | 2 | -0.047 (-0.337 to 0.243) | .75 | 0.130 (.72) | < 0.001 | ||
|
38 | N/A | |||||
From 1 day to 1 month | 24 | 0.194 (0.107 - 0.282) | < .001 | 39.329 (.02) | 41.519 | ||
Over 1 month to 4 months | 10 | 0.226 (0.089 - 0.363) | .001 | 7.139 (.62) | < 0.001 | ||
Over 4 months to 7 months | 4 | 0.157 (0.002 - 0.312) | .048 | 15.261 (.002) | 80.342 |
a Query 1
b Query 2
Forrest plot
The figures for intervention duration are presented in
Effect Size by intervention duration
To examine the long-term impact after an intervention had ended, all postintervention measures were grouped into 3 time categories. This resulted in the 38 distinct postintervention measures; these are referred to as Query 2 in
This section presents two analyses of psychological design. The first assesses the relationship between the overall psychological design and behavioral outcomes. The second analysis presents the psychological architecture of online interventions, reporting how frequently influence components are used and their associated effect sizes.
Of the theories used to design interventions, the transtheoretical approach was the most popular, being used across 47% (14/30) of the interventions. Other theories used to design interventions included social cognitive (4/30, 13%), cognitive behavioral therapy (4/30, 13%), behavioral therapy (3/30, 10%), extended parallel process model (2/30, 7%), health belief model (2/30, 7%), and the theory of reasoned action (2/30, 7%).
This section assesses relationships between an intervention’s overall psychological architecture and its effect size. The analysis is based on the coding systems of behavioral change techniques [
Groups of online interventions with the largest number of influence components demonstrated the largest effect sizes. Nonetheless, statistical correlations between influence components and effect size were inconclusive.
Sum of influence components by effect size
Meta-regression demonstrated a moderate but statistically insignificant correlation between an intervention’s total influence components and their effect size. However, there are reasons to suspect an association exists nonetheless. The meta-regression correlation between the sum of behavior change techniques and effect size is (
This section uses the CBICM as a framework to describe the psychological architectures employed by online interventions. Influence components are clustered within the social context, media channel, feedback message, source interpreter, source encoding, intervention message (behavior change techniques), and audience interpreter (behavioral determinants and demographics). To encourage personal change, many of the interventions helped participants adopt healthy habits by motivating them to set goals, record their behavior, learn new skills, and then use feedback to track their progress.
The
The
The
Media channel, feedback message, source interpreter, and source encoding
Absolute Coding | Relative Coding | ||||||||
CBICM clusters | k | % | k | d (95% CI) |
|
Qw ( |
I2 | ||
|
|||||||||
Website and email | 20 | 66.7% | 14 | 0.165 (0.054 - 0.276) | .004 | 24.914 (.02) | 47.820 | ||
Website | 10 | 33.3% | 8 | 0.309 (0.150 - 0.467) | <.001 | 16.636 (.02) | 57.922 | ||
|
|||||||||
Tailoring | 25 | 83.3% | 22 | 0.201 (0.107 - 0.296) | <.001 | 53.428 (<.001) | 60.695 | ||
Provide feedback on performance | 20 | 67.0% | 18 | 0.215 (0.109 - 0.321) | <.001 | 52.985 (<.001) | 67.915 | ||
Personalization | 12 | 40.0% | 11 | 0.193 (0.048 - 0.337) | .009 | 7.651 (.66) | < .001 | ||
Adaptation/content matching | 2 | 6.7% | 2 | 0.191 (-0.138 - 0.521) | .26 | 0.135 (.71) | < .001 | ||
|
|||||||||
Attractiveness | 5 | 16.7% | 3 | 0.080 (-0.215 - 0.375) | .60 | 2.631 (.27) | 23.975 | ||
Similarity | 3 | 10.0% | 3 | 0.324 (0.015 - 0.632) | .04 | 1.078 (.58) | < .001 | ||
Credibility | 1 | 3.3% | 1 | ||||||
|
|||||||||
Multiple interactions | 23 | 77.0% | 16 | 0.208 (0.098 - 0.319) | <.001 | 43.657 (<.001) | 65.641 | ||
Single interaction | 3 | 10.0% | 2 | 0.473 (0.154 - 0.792) | .004 | 0.001 (.98) | < .001 | ||
Sequential requests (foot in the door) | 1 | 3.0% | 1 |
Source intervention message (behavioral change techniques)
Absolute Coding | Relative Coding | ||||||
Behavioral Change Techniques | k | % | k | d (95% CI) |
|
Qw ( |
I2 |
Provide information on consequences of behavior in general | 23 | 76.7% | 16 | 0.306 (0.173 - 0.438) | < .001 | 11.365 (.72) | < 0.001 |
Goal setting (behavior) | 21 | 70.0% | 16 | 0.245 (0.131 - 0.359) | < .001 | 49.984 (< .001) | 69.991 |
Prompt self-monitoring of behavior | 19 | 63.3% | 16 | 0.223 (0.108 - 0.339) | < .001 | 52.600 (< .001) | 71.483 |
Provide instruction on how to perform the behavior | 18 | 60.0% | 15 | 0.212 (0.102 - 0.323) | < .001 | 27.927 (.02) | 49.870 |
Action planning | 17 | 56.7% | 13 | 0.240 (0.119 - 0.360) | < .001 | 46.702 (< .001) | 74.305 |
Provide normative information about others’ behavior | 12 | 40.0% | 12 | 0.246 (0.120 - 0.373) | < .001 | 6.893 (.81) | < 0.001 |
Fear arousal | 12 | 40.0% | 10 | 0.193 (0.042 - 0.344) | .01 | 6.491 (.69) | < 0.001 |
Barrier identification/problem solving | 10 | 33.3% | 10 | 0.224 (0.076 - 0.372) | .003 | 4.372 (.89) | < 0.001 |
Provide information on where and when to perform the behavior | 10 | 33.3% | 10 | 0.218 (0.095 - 0.340) | < .001 | 20.104 (.02) | 55.232 |
Set graded tasks | 10 | 33.3% | 9 | 0.095 (–0.017 to 0.207) | .10 | 11.464 (.18) | 30.216 |
Plan social support/social change | 9 | 30.0% | 5 | 0.250 (0.035 - 0.465) | .02 | 1.940 (.75) | < 0.001 |
Facilitate social comparison | 9 | 30.0% | 9 | 0.226 (0.070 - 0.382) | .004 | 4.439 (.82) | < 0.001 |
Model/demonstrate the behavior | 8 | 26.7% | 8 | 0.210 (0.056 - 0.365) | .008 | 3.886 (.79) | < 0.001 |
Provide information on consequences of behavior relevant to the individual | 8 | 26.7% | 8 | 0.208 (0.040 - 0.375) | .02 | 3.447 (.84) | < 0.001 |
Environmental restructuring | 7 | 23.3% | 4 | 0.189 (–0.028 to 0.406) | .09 | 1.229 (.75) | < 0.001 |
Prompt review of behavioral goals | 7 | 23.3% | 7 | 0.138 (–0.018 to 0.294) | .08 | 6.888 (.33) | 12.887 |
Agree behavioral contract | 5 | 16.7% | 4 | 0.275 (0.105 - 0.446) | .002 | 13.001 (.005) | 76.925 |
Prompt self-monitoring of behavioral outcome | 5 | 16.7% | 5 | 0.263 (0.080 - 0.446) | .005 | 36.961 (< .001) | 89.178 |
Prompt identification as role model/position advocate | 5 | 16.7% | 4 | 0.078 (–0.107 to 0.263) | .41 | 1.738 (.63) | < 0.001 |
Time management | 4 | 13.3% | 4 | 0.343 (0.018 - 0.669) | .04 | 1.476 (.69) | < 0.001 |
Stress management | 4 | 13.3% | 4 | 0.185 (–0.009 to 0.380) | .06 | 1.517 (.68) | < 0.001 |
Prompt self talk | 3 | 10.0% | 3 | 0.319 (0.058 - 0.581) | .02 | 2.168 (.34) | 7.747 |
Provide rewards contingent on successful behavior | 3 | 10.0% | 3 | 0.291 (0.023 - 0.560) | .03 | 1.478 (.48) | < 0.001 |
Provide information about others’ approval | 3 | 10.0% | 3 | 0.206 (–0.040 to 0.453) | .10 | 0.461 (.79) | < 0.001 |
Use of follow-up prompts | 3 | 10.0% | 3 | 0.183 (–0.098 to 0.463) | .20 | 0.968 (.62) | < 0.001 |
Goal setting (outcome) | 3 | 10.0% | 1 | ||||
Relapse prevention/coping planning | 3 | 10.0% | 2 | 0.149 (–0.100 to 0.398) | .24 | 0.310 (.58) | < 0.001 |
Shaping | 3 | 10.0% | 3 | 0.091 (–0.236 to 0.418) | .59 | 0.524 (.77) | < 0.001 |
General communication skills training | 2 | 6.7% | 2 | 0.295 (–0.031 to 0.622) | .08 | 2.737 (.10) | 63.458 |
Emotional control training | 2 | 6.7% | 2 | 0.253 (–0.061 to 0.568) | .11 | .796 (.37) | < 0.001 |
Prompting focus on past success | 1 | 3.3% | 1 | ||||
Prompt use of imagery | 1 | 3.3% | 1 | ||||
Motivational interviewing | 1 | 3.3% | 1 | ||||
Prompting generalization of a target behavior | 1 | 3.3% | 1 | ||||
Provide rewards contingent on effort or progress toward behavior | 1 | 3.3% | |||||
Teach to use prompts/cues | 1 | 3.3% | |||||
Prompt anticipated regret | 0 | 0% | |||||
Prompt practice | 0 | 0% | |||||
Prompt review of outcome goals | 0 | 0% |
The
Demographic moderators
Groupings | k | d (95% CI) |
|
Qb ( |
Qw ( |
I2 | |
|
30 | 1.248 (.74) | |||||
Younger (15.0 - 21.4) | 8 | 0.271 (0.095 - 0.446) | .002 | 4.676 (.70) | < 0.001 | ||
Middle (21.5 - 41.8) | 9 | 0.198 (0.045 - 0.352) | .01 | 4.725 (.79) | < 0.001 | ||
Older (41.9 and over) | 9 | 0.141 (–0.003 to 0.286) | .06 | 29.017 (< .001) | 72.430 | ||
Unknown | 4 | 0.190 (–0.033 to 0.414) | .10 | 8.196 (.04) | 63.397 | ||
|
30 | 5.889 (.12) | |||||
More female (66.6% - 100%) | 12 | 0.307 (0.187 - 0.427) | < .001 | 18.290 (.08) | 39.857 | ||
Mixed | 12 | 0.122 (0.010 - 0.235) | .03 | 11.354 (.41) | 3.116 | ||
More male (66.6% - 100%) | 2 | 0.123 (–0.111 to 0.357) | .30 | 0.864 (.35) | < 0.001 | ||
Unknown | 4 | 0.124 (–0.049 to 0.297) | .16 | 5.685 (.13) | 47.233 |
Audience interpreter (behavioral determinants)
Absolute Coding | Relative Coding | ||||||
Behavioral Determinants | k | % | k | d (95% CI) |
|
Qw ( |
I2 |
Knowledge | 30 | 100.0% | 16 | 0.291 (0.166 - 0.416) | < .001 | 53.257 (< .001) | 71.835 |
Motivation and goals (intention) | 26 | 86.7% | 20 | 0.229 (0.129 - 0.329) | < .001 | 54.332 (< .001) | 65.030 |
Social influences (norms) | 22 | 73.3% | 18 | 0.250 (0.147 - 0.354) | < .001 | 52.042 (< .001) | 67.334 |
Beliefs about consequences | 21 | 70.0% | 19 | 0.268 (0.182 - 0.353) | < .001 | 23.034 (.19) | 21.855 |
Skills | 19 | 63.3% | 15 | 0.185 (0.069 - 0.300) | .002 | 46.753 (< .001) | 70.055 |
Memory, attention, and decision processes | 18 | 60.0% | 17 | 0.188 (0.080 - 0.297) | .001 | 42.480 (< .001) | 62.335 |
Behavioral regulation | 17 | 56.7% | 14 | 0.218 (0.103 - 0.332) | < .001 | 40.971 (< .001) | 68.270 |
Emotion | 10 | 33.3% | 9 | 0.183 (0.026 - 0.341) | .02 | 6.966 (.54) | < 0.001 |
Nature of the behaviors | 9 | 30.0% | 6 | 0.274 (0.137 - 0.411) | < .001 | 16.142 (.006) | 69.024 |
Beliefs about capabilities (self-efficacy) | 8 | 26.7% | 7 | 0.083 (–0.051 to 0.218) | .23 | 4.545 (.60) | < 0.001 |
Environmental context and resources | 6 | 20.0% | 3 | 0.180 (–0.044 to 0.404) | .12 | 1.060 (.59) | < 0.001 |
Social-professional role and identity | 3 | 10.0% | 2 | 0.275 (–0.321 to .871) | .37 | 0.024 (.88) | < 0.001 |
To assess correlations among the 3 dose variables (intervention adherence, study adherence, and behavioral outcomes), 2 meta-analytical methods were employed and combined in
The first analysis pooled correlation effect sizes; is designated
In
Despite the two contradictory conclusions, there are compelling reasons why the relationship between intervention adherence and effect size is probably significant. Although the insignificant meta-regression analysis drew from more studies, the analysis was based on data that was heavily dichotomized, which is known to underestimate effect sizes [
Correlations between adherence variables and effect size (c = correlation effect size, m=meta-regression effect size)
Intervention duration, adherence, and behavioral outcomes
Study |
Intervention |
Behavioral Outcomes | ||||||
Intervention Duration | k | Average % | Weighted |
k | Average % | Weighted |
k | d (95% CI) |
Single-session | 3 | 73.9% | 72.9% | 2 | 100.0% | 100.0% | 4 | 0.404 (0.130 - 0.677) |
From 2 days to 1 month | 5 | 76.8% | 74.4% |
2 | 68.0% | 79.8% | 5 | 0.205 (0.026 - 0.383) |
Over 1 month to 4 months | 15 | 67.9% | 53.6% |
7 | 63.7% | 53.4% | 16 | 0.220 (0.116 - 0.324) |
Over 4 months to 7 months | 3 | 61.5% | 28.1% |
3 | 0.090 (–0.077 to 0.258) | |||
Over 7 months to 13 months | 2 | 66.8% | 68.0% | 2 | 43.0% | 42.3% | 2 | –0.047 (–0.337 to 0.243) |
The overall impact of online interventions is small, with the control conditions explaining much of the variance across studies. This suggests that online intervention efficacy should be regarded as a relative advantage in comparison to different intervention media. The largest impact was exerted from online interventions when compared with waitlists and placebos, followed by comparison with lower-tech online interventions; no significant difference was found when compared with sophisticated print interventions. In other words, online interventions offer a small effect and are probably as good as print interventions but with the advantage of lower costs and larger reach.
As a general guideline, an effect size d can be considered small (d ≤ 0.2), medium (d = 0.5), or large (d ≥ 0.8). Likewise, correlation effect sizes
Time proved to be a critical factor with shorter interventions achieving the largest impacts. In general, as the length of an intervention increased, behavioral impacts and intervention adherence decreased. When examining the long-term impacts after interventions had ended, the impact appeared to increase from 1 to 4 months and then decline afterwards. These trends may be partially explained by the relationship between adherence and behavioral outcomes, where the shortest interventions achieved both the highest behavioral impacts and also the highest levels of adherence. Discussed below, this trend is proposed to be a function of decreasing motivation.
Many of the interventions appeared to be simple but, in fact, were highly complex programs that used tailoring algorithms and which in some cases, contained libraries with potentially hundreds of messages that could offer thousands of message combinations. When designing interventions, the transtheoretical approach was the most popular theory used. Interventions were primarily goal orientated. In general, the interventions in this study informed users about the consequences of their behavior, encouraged them to set goals, then encouraged them to track their progress toward those goals while providing feedback on their performance. Popular behavioral determinants targeted by these interventions included knowledge, motivation, and social norms. Regarding demographics, younger audiences achieved the largest behavioral impacts, with impact strength decreasing as participants increased in age. Female dominated groups achieved larger behavioral outcomes in comparison with mixed gender and male dominated groups. Most interventions used feedback mechanisms, with 83% using tailoring, while the 40% that used personalization also combined it with tailoring. The most effective feedback mechanism was providing feedback on performance. Source factors were rarely reported; however, interventions that reflected similarity with users demonstrated efficacy. Just one intervention reported source credibility even though credibility has been recommended by numerous design guidelines [
Influence components approaches [
First, accurate relative coding of influence components could only take place when authors described the experimental and control groups in equal detail. Many authors did not fully describe control conditions, resulting in an overestimate of relative influence components, which may have caused measurement distortions. Additionally, interventions using stages of change frameworks tended to report a large number of influence components. However, depending on participants’ stage, they would likely be exposed to a smaller number of influence components, resulting in an overestimate in the number of relative influence components.
Second, the strong and statistically insignificant correlations found in this study suggest that this relationship may require a larger pool of studies to overcome measurement distortions. For instance, Webb et al [
Third, the relationship may not be linear but rather resemble an inverted u-shaped parabola curve. For example, one research team argued that websites that provide fewer individually tailored features may be more effective in promoting and maintaining behavior than ones that offer numerous poorly presented strategies [
Through absolute and relative coding, it was possible to examine an influence component’s frequency of use and associated effect sizes. In general, the frequency of use demonstrated a loose association with effect size. For instance, the most commonly used influence components were often the most effective ones, though there were exceptions to this rule. This suggest that, in general, intervention researchers are probably drawing from common approaches that have been proven to work, with a smaller amount of experimental work assessing less conventional approaches.
The law of attrition posits that study adherence and intervention adherence are likely to be correlated because they are impacted by a third variable, participant interest [
Instead of hypothesizing that attrition is a function of loss of participant interest, a slightly different proposal is that adherence is a function of participant’s motivation. By explaining the correlations as the result of motivation, this explains participant’s interest (in the terms of goal commitment) but also a second construct that encompasses ability and/or efficacy. Across different research, motivation generally encompasses these two dimensions: goal commitment and either self-efficacy or ability [
The law of attrition further proposes that study and intervention adherence follow a systematic pattern declining over time, similar to an inverse s-shaped diffusion curve [
Intervention length proved to be a critical factor, with shorter interventions generally achieving the largest impact and intervention impact fading as an intervention's length increased. This has implications for intervention designers who need to make interventions as short as possible to cope with rapid attrition and the probable loss of motivation over time. Moreover, for some behaviors, highly tailored single-session interventions produced the strongest effect sizes. This suggests that short and tailored interventions can be as effective, if not more effective, than some longer and demanding ones. However, this trend is likely to be limited to particular behaviors, such as responsible drinking [
Adherence variables demonstrated correlations with behavioral outcomes. This has implications for practitioners who generally seek to maximize behavioral impacts and researchers who must subject study participants to adequate dosage levels in order to conduct sound studies. To increase an intervention’s efficacy, it may be possible design adherence systems that encourage higher levels of intervention adherence. In some cases, interventions did not explicitly implement measures to maximize participant adherence, with 1 intervention attaining a median of 1 visit in 8 months [
By better understanding the components of motivation, promoters of healthy lifestyles can potentially design better interventions. Motivation is a likely explanation for the relationship between study adherence, intervention adherence, and behavioral outcomes. Intervention designers could potentially increase adherence by addressing the 2 common dimensions of motivation: participants’ goal-commitment and their ability/self-efficacy. For example, campaigns could benefit by intentionally designing online interventions around goals that appeal to the target audiences (following the social marketing approach), while also offering tailored support to aid participants who may lack ability or self-efficacy. Such an approach is similar to the Fogg behavioral model [
The capacity to develop mass-interpersonal online interventions may be limited by existing influence taxonomies that are not suitable to describing the psychological profile of interventions from an interpersonal or campaign perspective. During this study’s initial review of influence systems [
The scope of online interventions in this study is limited to those targeting voluntary behavioral change, similar to the types of interventions conventionally used in social marketing campaigns for public health. While coding influence components, some papers only provided vague descriptions, while others did not describe influence components other than those that comprise conventional therapy. It would have been ideal to code influence components directly from the interventions rather than research papers. Control conditions were rarely described in enough detail to code relative influence components with full confidence. As some influence components were used more often than others, this study may offer more reliable figures for popular influence components, which draw from a larger pool of studies. As there are few studies of online interventions targeting voluntary behaviors, it was necessary to combine effect sizes across behavioral domains. It would have been ideal to have at least 2 coders from which intercoder reliability calculations could have been estimated.
Although authors of similar meta-analyses have conducted numerous univariate analyses to assess effect sizes associated with moderator variables [
The studies in this meta-analysis demonstrate that online interventions targeting voluntary behavior change can work. Compared with waitlists, they demonstrate moderate efficacy, while compared with print materials, they offer similar impacts but with the advantages of lower costs and broader reach.
In general, the interventions informed users about the consequences of their behavior, helped them set and achieve goals, taught them skills, and provided normative pressure. Feedback mechanisms were common, with many interventions using tailoring along with personalization and offering services to track and report users’ progress toward their goals.
Motivation may be the critical factor that drives study adherence, intervention adherence, and impact. Time proved to be a critical factor, with impacts and adherence appearing to fade over time, perhaps as motivation depreciated.
Psychological design appears relevant to intervention efficacy. Although the relationships between the number of influence components and behavioral outcomes were inconclusive, there may be a relationship: Too few influence components may not be enough to influence behavior, while too many may be counterproductive. However, there may be a middle ground comprising a modest number of relevant influence components.
These findings suggest it is feasible to deploy online interventions that target individual-level behavior change, which can be scaled to achieve population-level health benefits. Given the high-reach and low-cost of online technologies, the stage may be set for increased social marketing campaigns that blend mass-media outreach with interpersonal digital support. For example, this means fewer public health campaigns that just disseminate warnings or advice and more campaigns that offer online tailored support in the form of digital therapists that help citizens help themselves.
None declared
Communication-based influence components model
analysis of variance
communication-based influence components model
confidence interval