This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Web-based interventions for problem drinking are effective but characterized by high rates of attrition. There is a need to better understand attrition rates in order to improve the completion rates and the success of Web-based treatment programs.
The objectives of our study were to (1) examine attrition prevalence and pretreatment predictors of attrition in a sample of open-access users of a Web-based program for problem drinkers, and (2) to further explore attrition data from our randomized controlled trial (RCT) of the Web-based program.
Attrition data from two groups of Dutch-speaking problem drinkers were collected: (1) open-access participants enrolled in the program in 2009 (n = 885), and (2) RCT participants (n = 156). Participants were classified as noncompleters if they did not complete all 12 treatment sessions (9 assignments and 3 assessments). In both samples we assessed prevalence of attrition and pretreatment predictors of treatment completion. Logistic regression analysis was used to explore predictors of treatment completion. In the RCT sample, we additionally measured reasons for noncompletion and participants’ suggestions to enhance treatment adherence. The qualitative data were analyzed using thematic analysis.
The open-access and RCT group differed significantly in the percentage of treatment completers (273/780, 35.0% vs 65/144, 45%, χ2
1 = 5.4,
The challenge of Web-based alcohol treatment programs no longer seems to be their effectiveness but keeping participants involved until the end of the treatment program. Further research should investigate whether the suggested strategies to improve adherence decrease attrition rates in Web-based interventions. If we can succeed in improving attrition rates, the success of Web-based alcohol interventions will also improve and, as a consequence, their public health impact will increase.
International Standard Randomized Controlled Trial Number (ISRCTN): 39104853; http://www.controlled-trials.com/ISRCTN39104853 (Archived by WebCite at http://www.webcitation.org/63IKDul1T)
Web-based interventions for problem drinkers improve the availability of alcohol treatment services and reach a more diverse segment of the population of problem drinkers [
In his law of attrition, Eysenbach distinguished two processes of attrition: dropout attrition and nonusage attrition [
Usage and follow-up completion rates of Web-based alcohol interventions studies published to date range from 16.5% [
Attrition data have been mainly coming from trials. Compared with the dropout and nonusage attrition rates in effectiveness trials of Web-based interventions, attrition rates in open-access interventions are higher [
The high percentages of nonusage attrition lead to the question of whether Web-based alcohol treatment might work more effectively for some people than for others. Exploring the variables that make individuals more vulnerable to not completing treatment may help us to identify target groups and develop strategies to address the nonusage attrition problem. We examined three types of variables that were associated with nonusage or dropout attrition: sociodemographic variables, drinking behavior, and psychological variables. Those factors have been investigated in several online alcohol intervention studies. Although most studies found no differences in baseline variables between completers and noncompleters [
None of the Web-based alcohol intervention studies formally examined the reasons for noncompletion. Although most studies report the rates of nonusage or dropout attrition, they do not report the reasons for attrition. However, in our recently conducted RCT we examined the reasons for not completing treatment [
The first aim of this study was to examine attrition prevalence and pretreatment predictors of attrition in a cohort of open-access users of the Web-based treatment program. The second aim was to further explore attrition data from our RCT. We investigated the prevalence of attrition, the reasons for noncompletion, pretreatment predictors of attrition, and participants’ suggestions for how to enhance treatment completion. Accordingly, the present study allowed us to compare the attrition data of both samples: a trial and an open-access group of users.
The real-world sample consisted of all open-access users of the Web-based alcohol treatment program in 2009 (n = 885). The only inclusion criterion for open-access users was a minimum age of 18 years. All data entered by participants were stored in the Web-based application. We could identify who accessed the Web-based treatment program and who did not, the duration of participation for treatment completers, and the number of completed sessions in case of noncompletion. Participants who dropped out were not assessed about their situation at that time; because of the feasibility nature of the open-access study and the linear design it was not possible to send questionnaires to nonresponders through the application.
We conducted secondary analyses of our RCT: an open trial with participants randomly assigned to either the Web-based treatment group or to the waiting list control group [
The Web-based alcohol treatment consisted of a structured, 2-part, online treatment program in which the participant and the therapist communicated asynchronously, via the Internet only. The method underlying the program was based on the principles of cognitive behavior therapy [
Participants’ pretreatment characteristics were derived online from the baseline self-report questionnaire, for RCT as well as for open-access participants. Weekly alcohol consumption was assessed by a 7-day retrospective drinking diary, including a question about atypical drinking [
The outcome measure of the logistic regression analysis was completion of the Web-based alcohol treatment program; this was defined as completion of all 12 treatment sessions: 9 assignments and 3 assessments. Because of the linear design of the treatment program it was impossible for participants to skip parts of the intervention; therefore, the point at which they stopped using the program indicates exactly how much treatment participants received. In our study nonusage attrition automatically meant dropout attrition and we will therefore just use the term attrition.
In order to gain insight into the motives of participants to stop using the Web-based treatment program, noncompleters in the RCT group received an email with a link to an additional online questionnaire consisting mainly of open questions concerning their perception of the program, reasons for discontinuation, and suggestions to improve the intervention and enhance treatment completion. If participants did not complete this questionnaire, they were contacted by telephone to remind them to complete the questionnaire either online or alternatively by phone.
Chi-square and
Reasons for nonusage attrition were independently assessed by the first and third author (qualitative study). The agreement level between both authors was 87%, which was considered acceptable. If the two authors did not agree, the topic was discussed in order to reach agreement. Participants’ responses to open questions were analyzed using thematic analysis. The first author carefully searched through the data to identify and code all features concerning participants’ reasons for not completing the treatment program. After collating relevant data with each code, related patterns were combined into themes. After refining and defining the themes, a brief description of each theme was formulated related to the research questions of the study.
Of the 885 registrants for the open-access version in 2009, 105 never started using the Web-based alcohol treatment program by doing the first assignment, sending a message to their therapist, or logging into the daily alcohol diary. Of the 780 participants who started the open-access version, 54.0% (n = 421) were women, 49.6% (n = 387) had a higher education level, and 69.0% (n = 538) were employed. Age ranged from 20 to 78 years, with an average of 45.7 years (
Characteristics of participants in the randomized controlled trial (RCT) and open-access group
Variable | RCT participants |
Open-access participants |
|||
n | % | n | % | ||
Female | 83 | 58 | 421 | 54.0 | |
Higher education | 84 | 58 | 387 | 49.6 | |
Employed | 117 | 81.3 | 538 | 69.0 | |
|
|||||
Alcohol dependence | 120 | 83.3 | 684 | 87.7 | |
Alcohol abuse | 14 | 10 | 42 | 5 | |
No dependence or abuse | 10 | 7 | 54 | 7 | |
Prior treatment for alcohol abuse | 22 | 15 | 226 | 29.0 | |
Prior treatment mental health problems | 72 | 50 | 455 | 58.3 | |
|
144 | 100 | 689 | 88.3 | |
Mean | SD | Mean | SD | ||
Age (years) | 45.8 | 9.7 | 45.7 | 10.8 | |
|
|||||
Men | 49.8 | 26.9 | 49.1 | 30.1 | |
Women | 32.6 | 14.6 | 37.3 | 22.9 | |
GHQ-28 scorec | 52.6 | 11.9 | NAd | NAd | |
MAP-HSS scoree | 19.8 | 6.2 | NAd | NAd | |
|
|||||
Depression score | 8.7 | 8.4 | NAd | NAd | |
Anxiety score | 5.9 | 5.9 | NAd | NAd | |
Stress score | 12.5 | 8.2 | NAd | NAd | |
|
|||||
Precontemplation | 12.1 | 1.3 | 12.3 | 1.6 | |
Contemplation | 17.1 | 2.1 | 17.1 | 2.3 | |
Action | 12.4 | 3.5 | 13.3 | 3.3 | |
|
|||||
Treatment Readiness | 4.0 | 0.5 | 4.1 | 0.4 | |
Desire for Help | 3.9 | 0.7 | 3.9 | 0.6 |
a
b Drinking >21 (men) or >14 (women) mean units per week.
c 28-item General Health Questionnaire.
d Not applicable.
e Maudsley Addiction Profile-Health Symptom Scale.
f 21-item Depression Anxiety Stress Scale.
g Readiness to Change Questionnaire.
h TCU Motivation for Treatment scale.
Flow of participants in the randomized controlled trial.
Of the 780 open-access participants, 65.0% were noncompleters. Treatment completers (n = 273, 35.0%) completed all 12 treatment sessions and noncompleters (n = 507, 65.0%), an average of 4.8 (SD 3.1) sessions. Of the 144 RCT participants, 55% were noncompleters. Treatment completers (n = 65, 45%) completed all 12 treatment sessions and noncompleters (n = 79, 55%), an average of 4.8 (SD 3.1) sessions. The open-access and RCT group differed significantly in the percentage of treatment completers (χ2
1 = 5.4;
Participants completed the sessions in the order that they were presented. The average duration of treatment to completion was 16.1 weeks in the RCT sample and 17.1 weeks in the open-access sample.
Attrition curve: proportion of participants by number of assignments in the randomized controlled trial (RCT) and open-access group.
We found only one significant difference between completers and noncompleters in the RCT sample. The mean score on the Treatment Readiness subscale of the MfT was higher for completers (mean 4.13) than for noncompleters (mean 3.97),
We found seven significant differences between completers and noncompleters in the open-access sample: age, gender, education level, baseline alcohol consumption, prior mental health treatment, treatment readiness, and readiness to change action score. The differences are shown in
Differences between open-access completers and noncompleters
Variable | Completers (n = 273) | Noncompleters (n = 507) | Test result | ||||
n | % | n | % | χ2 | df |
|
|
Female | 170 | 62.3 | 251 | 48.5 | 11.6 | 1 | <.001 |
Higher education | 163 | 59.7 | 224 | 44.2 | 17.1 | 1 | <.001 |
Prior mental health treatment | 175 | 64.1 | 280 | 55.2 | 5.8 | 1 | .02 |
Mean | SD | Mean | SD |
|
df |
|
|
Age (years) | 47.8 | 10.4 | 44.5 | 10.9 | –4.14 | 1,778 | <.001 |
Baseline alcohol consumption (standard units/week) | 37.4 | 24.0 | 45.6 | 28.2 | 4.05 | 1,778 | <.001 |
MfTa Treatment Readiness | 4.1 | 0.4 | 4.0 | 0.4 | –3.30 | 1,423 | .001 |
RCQb action score | 13.8 | 3.3 | 13.0 | 3.3 | –3.43 | 1,778 | <.001 |
a TCU Motivation for Treatment scale.
b Readiness to Change Questionnaire.
Multivariate logistic regression analysis without treatment readiness (n = 780) revealed a statistically significant contribution of age, gender, education level, baseline alcohol consumption, and readiness to change action score. Predicted probabilities of the model of x, y, and z led to a specificity of 85% with a sensitivity of 25%, a specificity of 80% with a sensitivity of 35%, and a specificity of 75% with a sensitivity of 43%, respectively. The Nagelkerke
We divided noncompleters into early and late noncompleters to determine whether the two groups differed. We considered noncompleters who completed a maximum of 3 assignments to be early noncompleters and those who completed at least 4 assignments to be late noncompleters. We found no differences between both groups in the RCT sample (n = 144). However, in the open-access sample (n = 780) we found that, compared with those who completed fewer assignments, more noncompleters who completed at least 4 assignments had a high level of education (128/221, 57.9% vs 93/221, 42%, χ2
1 = 6.1,
A diversity of personal reasons were given as reason for noncompletion (n = 22), including being too busy with work, a seriously ill family member or bereavement, other priorities, a hospitalization, no Internet access, or moving house.
Participants who identified the Web-based alcohol intervention itself as a reason for discontinuation (n = 17) most commonly indicated that the program was too time consuming or too demanding. Some participants reported that the program could not meet their personal needs.
Several participants reported that they no longer felt the need to continue the program, because of the progress they made (n = 11). They gained from the intervention what they needed and felt in control of their drinking behavior.
For 2 participants the Web-based treatment program was only the first step in working on behavioral change, and they continued treatment in a face-to-face setting. Of the persons whose formal reason for dropout is unknown (n = 18), the messages in their personal records provide some information. Participants mentioned several times that working on their alcohol problem was quite confrontational and overwhelmed them too much. Some participants also reported more or less lack of motivation.
Several RCT participants gave suggestions as how to improve the Web-based treatment program. One of the suggestions was sending an email message to participants to notify them that they had received a new message or assignment from their therapist. This it was felt would act as a reminder and prevent unnecessary logging into the application. Another suggestion was to allow more flexibility in the treatment protocol, with the possibility of skipping sessions when required—for example, the possibility to start immediately with the goal-setting assignment or no longer mandating daily registration. In its current form it was not possible to move on to the next assignment without completing the previous one. Some participants also mentioned the need for additional contact: the choice to contact their therapist by phone or face-to-face and the chance to get in touch with fellow participants, with the suggestion to link each participant to his or her own buddy. Some participants made suggestions for improving the usability of the Web-based treatment program, including the speed of the intervention, layout characteristics, and button functions.
The aim of this study was to explore the attrition data of an open-access and an RCT sample of a Web-based treatment program for problem drinkers. The study demonstrated high prevalence of attrition in both samples, with 10% less treatment completers in the open-access sample. Participants’ readiness for treatment, gender, education level, age, baseline alcohol consumption, and readiness to change score were shown to predict treatment completion. The key reasons for noncompletion were personal reasons, dissatisfaction with the intervention, and satisfaction with their own improvement. The main suggestions for boosting strategies involved email notification and more flexibility in the intervention.
Attrition was high in both samples. Although our attrition rates of 65% in the open-access sample and 55% in the RCT sample are in line with those found in other Web-based alcohol intervention studies [
The variety of nonusage and dropout attrition rates in Web-based alcohol interventions is relatively similar to that found in Web-based treatments for psychological disorders, ranging from 2% to 83% [
In both study samples, the pattern of nonusage attrition was steady throughout the intervention period. This means that both groups showed the same trend of attrition; at each treatment session participants dropped out. The number of dropouts gradually decreased, regardless of whether participants participated in the RCT or in the open-access intervention. Although the gradual decrease is in contrast with the suggestion of Eysenbach [
The percentage of dropouts seems to be the highest after session 3, concerning the daily drinking diary assignment. A possible explanation might be the intensity of this assignment, as participants have to register their alcohol consumption every day. This might be quite confrontational and participants might also feel uncomfortable or annoyed by daily registration of their drinking amount.
The differences we found between early and late noncompleters prove that noncompleters who completed at least 4 assignments were more similar to treatment completers than they were to those who completed fewer than 4 assignments.
The only statistically significant predictor of treatment completion in the RCT sample was a higher treatment readiness score, measured by the Treatment Readiness subscale of the TCU MfT questionnaire. In the open-access sample, higher treatment readiness also was a significant predictor, as were higher age and lower baseline alcohol consumption when the treatment readiness variable was included (n = 425). In the open-access sample without the treatment readiness variable (n = 780), the statistically significant predictors were higher age, female gender, higher education level, lower baseline alcohol consumption, and higher readiness to change action score. Other factors were found to have no predictive value.
Based on our different findings in the three subsamples and in line with an analysis of the literature by Melville and colleagues [
In addition to the quantitative data of the RCT and open-access sample, the qualitative data provided more insight into the reasons for noncompletion and the possibilities to reduce potential loss. The present more extensive findings confirm the earlier findings on dropout from our RCT study and, as discussed before [
Boosting strategies are desirable to maximize the number of treatment completers in trial settings as well as in open-access interventions. Participants themselves suggested sending email reminders as an additional supportive resource. The use of push reminders, such as phone calls, postcards, and email messages, previously has shown improved treatment completion rates [
This study has several limitations that are important to acknowledge. Due to the technical structure of our intervention, noncompletion included not just stopping using the intervention but also no longer receiving posttest and follow-up assessments. The therapists and participants could not move on to the next assignment or questionnaire without completing the previous one. We chose this linear model because of the protocolled treatment and the preference for completing treatment steps in strict order, to ensure best quality and that the questionnaires would be completed. However, a consequence that we have not sufficiently taken into account is that nonusage attrition also meant study attrition and that we unfortunately never obtained a lot of data from noncompleters. This is definitely not desirable and needs to be changed in future studies. One of the consequences is that we did not have data available to compare treatment outcome of completers versus noncompleters. Although our qualitative data indicated that completers had better treatment results, this assumption can be confirmed only with quantitative data. As far as we know, no previous online alcohol intervention study has investigated the difference in treatment outcome between completers and noncompleters. We therefore recommend investigating the impact of compliance on treatment outcome in future studies.
We also decided not to use push factors in our RCT to keep the trial setting as natural as possible. However, it is possible that, if we had used push factors, we could have raised the response rate to generate a more complete dataset.
Another limitation is that only baseline characteristics were considered as potential predictors of treatment completion. It is possible that other factors such as forum use or the therapeutic relationship also influenced attrition rate. However, we aimed to determine at baseline which participants would complete the whole treatment program. We were also limited to the baseline characteristics we measured and therefore not able to include some of the variables previously found to have predictive value.
Both study samples consisted largely of adults in their mid-40s. This can partly be explained because our samples consisted of problem drinkers from the general public. And although we previously found that the average age of face-to-face clients was slightly lower, face-to-face clients also have a mean age of around 43 years [
Nowadays, the challenge of Web-based alcohol treatment programs no longer seems to be their effectiveness but keeping participants involved until the end of the treatment program. Our study provided some points that therapists might focus on, including helping participants to be ready for treatment and for change. We should also investigate the effect of starting immediately with reduction of alcohol consumption. Boosting strategies such as email notification and more flexibility in the intervention might also help to improve adherence. Further research should investigate whether those changes lead to decreased attrition rates in Web-based interventions. If we can succeed in improving attrition rates, we assume that the success of Web-based alcohol interventions will further improve and, as a consequence, they will have a greater public health impact.
This study was funded by Tactus Addiction Treatment and the Nijmegen Institute of Scientist-Practitioners in Addiction.
None declared
confidence interval
Depression Anxiety Stress Scale
Diagnostic and Statistical Manual of Mental Disorders, 4th revision
28-item General Health Questionnaire
Maudsley Addiction Profile-Health Symptom Scale
TCU Motivation for Treatment scale
Readiness to Change Questionnaire
randomized controlled trial
receiver operating characteristic