Early last year, Eysenbach published a paper [
Eysenbach’s paper gives us three major conceptual advances – the distinction between trial dropout and low/nonusage nondropouts; the proposition of “diffusion of innovation” effectively reversed as a model for the take-up of Internet interventions; and the concept of the “Run in and Withdrawal Design.” The diffusion of innovation reversed is essentially a structural one in that it suggests a number of “systems” features that influence dropout and usage including expectation management, ease of ease of enrolment, ease of dropout, usability, adjunct personal contact, financial commitment, workload, competing events, and experience.
A number of issues arise from Eysenbach’s proposal. First, the structural or systems model factors in the model may need to be supplemented by consideration of user characteristics. For example, the use and uptake of Web sites in mental health are likely to be modulated by the severity of the user’s mental health problem [
A second attribute of users that warrants incorporation in any model of nonusage is an understanding of the expectations that people bring to a Web site, and what they mean by their intention-to-use. For example, many young people do not recognize “lousy feelings” as depression or anxiety, but a brief visit to a Web site provides a “mini-diagnosis” and a label. For them, one module may well fulfill their needs: They have no expectation that they are lining up for a full set of modules. Recognizing these multiple paths and trajectories of web usage means that low usage and dropout do not necessarily coincide with “failure”. Dropouts may well be e-attainers [
The multiple uses made of Web sites by different users raises the distinct, but highly relevant issue of the suitability of the Internet to provide full treatment packages for different conditions. The Internet has the capacity to reach many individuals who may never seek formal treatment for mental health services. However, it may well be that the primary role of the Internet in disease prevention will be in the delivery of short positive health messages, rather than the delivery of ‘therapy’ that requires hours of online work. Web site adherence or “stickiness” may cease to be an issue for online sites like MoodGYM when shorter interventions can be demonstrated to lead to similar health outcomes and brief bursts of information lead to increased help-seeking.
The following is an example of how attrition may be influenced by the personal characteristics or the preferences of the online users. We are currently conducting a trial of MoodGYM in which those intending to use the program: (a) report that they have been asked to do the trial as a part of their clinician’s treatment plan; (b) chose to do five modules when offered the opportunity to do only fewer than five in the early part of MoodGYM; or (c) are randomized to the MoodGYM condition as a function of an ongoing trial.
Completion rates of MoodGYM modules as a function of group membership
We suggest that Eysenbach’s argument asserting the differences between e-health and traditional clinical trials might be slightly overstated. Many researchers who have been involved with traditional randomized controlled trials (RCTs) of pharmacological treatments in psychiatry will recognize Eysenbach’s characterization of attrition in such settings as extremely optimistic. The dropout of a third of recruited patients in such trials is common with rates exceeding 80% being observed in some long term trials aimed at relapse prevention. Determining whether patients have complied with medication regimes is difficult. In many respects, e-health is in a far stronger position than other studies to detail the low usage of the interventions, given the tracking of length and number of visits to the application. Moreover, e-health interventions have high fidelity: the exact same intervention is potentially available to all the participants.
There are other minor points that need to be made. For example, Eysenbach makes a distinction between users lost to dropout and low usage nondropouts. This model suggests that people discontinue innovations because they are disenchanted or because they seek a better alternative to meet their needs. On reflection there are four theoretically possible usage curves: (i) dropout, low or no usage; (ii) nondropout, low or no usage; (iii) dropout, high usage; and (iv) nondropout, high usage. The dropout, high usage is a person who prefers not to engage with a Web site but undertakes the program under a new user name each visit (if the application is an open web-based one).
Up until recently, interventions and clinical trials have been analyzed using classical analysis of variance methods. For these techniques, missing observations arising from participant dropout are a just nuisance factor which is addressed,
Mixed models themselves throw little light on the nature of attrition, its causes or consequences. However, more advanced techniques, based on latent variable modeling, may help us understand the complexity of the multiplicity of paths through and of out interventions. The complier average causal (CACE) model is specially aimed at estimating the effect of an intervention in the presence of noncompliance [
Developing a metric of the attrition attributable to an internet intervention site is an attractive initiative. It would parallel the notion of the acceptability or tolerability of conventional treatments. This concept, usually measured informally or only crudely, recognizes that some treatments, while efficacious, are possibly so odious as to be persevered with by only a few patients who might benefit from them. There are substantial hurdles to such measurements. It will prove difficult, if not impossible to disentangle attrition due to site effects from attrition due to the characteristics of users and the paths they take to a site. More important, to focus exclusive on attrition is to focus on the negative side of e-health interventions. E-health interventions have enormous potential to reach those warranting assistance and to address their needs. Recognizing the fact of high attrition, we need to respond with a science (and an art) of participation and encouragement.
This research was partially supported by an NH&MRC project grant (no. 332950) to Andrew Mackinnon.