The past two decades have seen a shift from traditional face-to-face consultations to technology-driven interventions or e-therapies. Recent reviews have shown that e-therapies are both effective [1-3] and growing in popularity. This is supported by the level of publications relating to e-therapies: Medline and PsycINFO citations in the subject group “online therapy” rose from 12 citations between 1991 and 2000 to 709 citations from 2001 to September 2010.

A potential difficulty in evaluating these programs is adherence (see [4]). Little is known about the degree to which users’ engagement matches the usage pattern for which the websites are designed. Also, little is known about the influence of program adherence on outcomes. Within the medication literature, adherence, “the extent to which a person’s behaviour – taking medication, following a diet and/or executing lifestyle changes, corresponds with agreed recommendations from a health care provider” [5], and persistence, the act of adhering to treatment recommendations for the prescribed duration of time [6], are widely studied. These behavioral variables significantly influence medical [5] and psychotherapy [7-9] outcomes. In pharmaceutical trials, a dose–response curve is often plotted to understand the optimal level of medication to reach a desired response, and adherence is considered highly influential within this. In e-therapy, adherence may be just as important a consideration.

The eHealth equivalent of failing to persist with therapy is treatment dropout. Treatment dropout refers to when a user prematurely stops using the intervention. Some of these users may remain in the trial, completing the trial assessments, while others may choose to leave the trial. Those who choose to leave the trial early are said to have discontinued and are reflected in trial attrition rates. Such attrition can affect the ability of results to be generalized [10,11] and it undermines the statistical power of the trial. Many authors note that dropout rates are high, particularly in open-access trials [12-14] where the intervention is made available to the public with minimal or no cost. Entry into these trials is therefore open, with users being able to join at any time. Despite each user’s choice to engage with the site, only a small proportion of users persist with the trial and associated follow-ups. However, with some programs, these figures may still be comparable to [15], or even lower than [16], the dropout rates found in traditional face-to-face therapies. Little is known about the impact of the degree of persistence on outcome in those who complete trial ratings and outcomes.

While an appreciation of persistence is important in evaluating e-therapy, an understanding of adherence to the program content, such as the completion of program modules or online activities, may be more so. As the field of e-therapy has been growing, so has the interest in potentially modifiable user factors that may influence adherence. Clearly, program content evolves from empirically supported research, but only recently have the exploration and manipulation of program factors become foci. Program usability and feasibility testing is increasingly common. Recent findings have indicated that greater use of computer relational skills, such as the use of empathy and social dialogue in the computer program, leads to increased program usage [17]. Many trials use pilots and usability studies to ensure the program functions as planned [18-22]. Several authors have begun to explore the impact of reminders and trial factors on dropout. Clarke and colleagues’ [23] comparison of the Overcoming Depression on the Internet websites found that participants were more likely to use the program as recommended if they received reminders, and that this increased use was present regardless of the type of reminder (telephone versus postcard) that they received. Alternatively, Christensen et al [24] demonstrated that weekly tracking and reminders reduced attrition in a cognitive behavior therapy e-therapy intervention for depression. Christensen et al [25] also found that shorter cognitive behavior therapy e-therapy interventions were not as effective as longer interventions but that attrition rates were lower, potentially indicating an important trade-off between acceptability and effectiveness. It is thought that the variation in adherence and persistence may be due to the participant’s discretion in using e-therapy [26], rather than therapy that is prescribed, as in a drug trial [13]. Engagement in an e-therapy trial tends to require more physical, cognitive, and time investments by the participant, compared to the relative ease of taking a medication daily. This may explain why e-therapy users are more prone to dropout and nonadherence.

E-therapy trials have an advantage over traditional trials when measuring persistence and adherence, in that more objective measures of intervention usage are readily available to researchers. Objective measures of persistence include metrics such as the number of times the participant accesses the program, and adherence measures include the time spent online, number of completed activities, and patterns of usage. Subjective measures, such as estimated time spent online, reporting on the completion of behaviorally based homework activities, and use of skills, can also be incorporated. Despite the relative ease of capturing these data in online interventions, few studies report these statistics. Merely reporting on adherence and dropout provides limited insight about the impact of adherence on program outcomes or the translation of program behaviors into daily life. Even fewer studies examine the role of adherence on outcomes. The few that do, by publishing only significant results, may result in publication bias.

Recent studies have begun to address this through the exploration of the relationship between program exposure and outcomes [27-30]. Although more data can often be collected in e-therapy trials, there is a need to consider what data are collected, how they are collected, and how adherence is defined. It is likely that the influence of design, application, and logistics of e-therapies on outcomes can become informed in the same way medication dosing affects outcomes. For instance, single daily dosing or polypills have been suggested as methods of improving adherence and outcomes in medication treatments.

To inform this development, we conducted a systematic review of the literature evaluating adherence in e-therapies. Within the context of this review, adherence is defined as the degree to which the user followed the program as it was designed. The two aims were (1) to review the methods used to assess adherence and (2) to evaluate the association between adherence measures and outcome.

We initially found a total of 8300 studies by using the above combination of search terms. Once duplicates were removed, the articles were further limited to those that were published in an English-language, peer-reviewed journal (n = 1095). The remaining articles were systematically reviewed (by LD) to ensure they met the inclusion criteria. This resulted in a total of 74 articles describing 69 individual studies. The heterogeneity of adherence and outcome measures precluded a formal meta-analysis.

Sample size varied significantly between the trials, ranging between 20 and 3176 (median 146), with many of the open-access trials having more participants (Table 1). Structured, discrete-period clinical trials tended to have fewer participants. This may be due to the seemingly more intensive researcher input in the form of feedback provided by emails, monitoring of activity completion, and moderation of activity, requiring greater resourcing for smaller numbers of participants. Larger trials tended to evaluate unstructured websites where participants were able to use the website in whatever manner they pleased, rather than using a structured preplanned program.

Mean study discontinuation rate was approximately 23% of all trial participants (range 0%–83%). Studies of physical and psychological target behaviors had similar attrition rates.

Understanding adherence to e-therapies is important in understanding how these therapies may benefit individuals who need intervention. The impact of adherence on outcome appears to vary. The review demonstrates that however it is measured, adherence is associated positively when reported with intervention outcomes targeting physical health. However, in the most commonly targeted outcome of e-therapy, depression, the number of logins, self-reported activities, time online, and pages opened showed no such association. Only 2 measures of adherence—the degree of completion of the modules within the program and a summative “website exposure” outcome—were associated with better depression outcomes.

The association between the number of logins and outcomes in e-therapies targeting physical health is similar to the positive relationship observed between session attendance and outcome in physical rehabilitation [67-70] and to the dose–response curve seen in medication therapy. This association may largely be due to the number of logins being representative of the participant’s willingness to use the program through their return to the website. The number of logins may be more indicative of program usage than are self-reported activities completed or forums posted. This is particularly the case in programs where there is structured program use (therefore needing completion of an activity or module prior to moving to another) but no time restrictions placed on progression. In this style of program, a participant may complete several modules or activities during a single login. Therefore, the benefits received in completing a module, processing it over a time period, and using its skills before building on this with the next module may be lost.

For participants involved in trials targeting weight changes as an outcome [38-40], all measures of adherence were correlated with outcomes. Therefore, more adherent participants had a higher level of weight loss. This is consistent with the current literature, which shows that the more closely people follow dietary plans, the better their outcomes [71]. In the case of physical outcomes, one interpretation of the results could be that adherence is a marker of an unmeasured factor such as personality or self-efficacy that predisposes participants to a better outcome. A more extreme interpretations is that the content of the intervention is irrelevant; it is the application of the participant to “something” that produces the outcomes.

The most common use of e-therapy is to intervene in depression and anxiety. However, measures of logins, self-reported activity completions, and time online were not associated with outcome—only evidence of actual module completion either on its own or in the composite “program exposure” measure appeared to be related to outcome. This suggests a more nuanced interpretation, in that a participant’s interaction with the module content leads to change, rather than improvement merely reflecting a greater propensity to adhere. This reflects the face-to-face literature, in that the structured psychotherapies, with module (task) completion and review, are generally more effective than supportive psychotherapy [72-74]. The lack of association of many adherence measures with outcome might be explained by lack of power in these studies. This is particularly supported by more consistent findings in physical health interventions, which had a median sample size nearly twice that of psychological interventions. However, it is of note that many psychological health studies did have large sample sizes and, for some adherence measures, the results were not null, but negative.

The definition of adherence and its consequent analysis also varied considerably across studies. Several studies categorized participants into adherent or not [49,54,57,75], or usage categories such as low, medium, and high [60], while others used continuous variables [23,24,33-36,38-41,43-45,50-52,55,56,58,59,61,76] or a combination of these [42,46,53,64-66,77]. Such variation is likely to produce mixed findings. It is also worth noting that several studies reported collecting multiple measures of adherence but reported on the outcome of only a few selected items. In many, it is unclear why some measures of adherence were reported over others, though this may presumably reflect a reporting bias and may therefore inaccurately portray the role of adherence. It is possible that unreported relationships may be nonsignificant, but may also have been useful in understanding the important components of program engagement and may build understanding of the user-engagement factors of e-therapy.

The variation in measurement of adherence in online interventions makes it difficult to accurately determine the impact of adherence on outcome. Despite having several seemingly objective measures such as number of logins, time spent online, and activities completed, there are still difficulties in determining the “dose” a participant receives. Specifically, time spent online is likely to be influenced by factors such as processing speed, cognitive ability, reading aptitude, and familiarly with using computers, several of which are likely to be influenced by psychological health. Likewise, a person who is unfamiliar with using a computer may write out their activities, rather than engage in the functionality presented in the online environment. This needs to be explored further by studies that randomly assign participants to receive different doses of programs, such as that by Christensen et al [25], which explored exposure to modules against change in outcome measures. Future interventions that intend to measure the impact of adherence on outcomes would benefit from clearly defining their adherence variables and exploring all relationships between the potential adherence variables and outcomes. In addition to this, measures of inactivity need to be included where temporal measures are used. Therefore, when a measure of adherence is the amount of time spent online, there needs to be a time-out function of need to engage readily with the program, to show that it is being used. This will allow the exclusion of individuals who have left their computer on and who are no longer engaging with the program material.

From this review, the mechanism of how adherence influences Internet treatment outcomes is unclear. Medical studies have shown that adherence is associated with better outcomes, regardless of whether patients received placebo or active interventions [78,79]. Within this review, it is unclear whether adherent patients generally do better, regardless of condition. It has been suggested that the mechanisms underlying a generalized adherence effect may be similar to the mechanisms that underlie the placebo effect [80], whereby the positive outcomes achieved from taking a placebo are the result of internalized beliefs about therapy, such as the expectations that the individual holds or the belief that the therapy will be effective. The placebo effect may also indicate a phenomenon of regression to the mean, in that outlying behavior returns closer to the mean over time. Alternatively, it has been suggested that adherence may be a more general indicator of orientation to healthy behaviors. Therefore, individuals who are adherent may be more likely to follow healthy lifestyle practices and therefore have improved outcomes [81,82]. Further exploration within the area of e-therapy needs to determine whether adherence influences outcomes through expectation beliefs or through participants being generally adherent to treatment recommendations. To do this, control group adherence behaviors need to be reported more.