This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Computer-delivered psychological treatments have great potential, particularly for individuals who cannot access traditional approaches. Little is known about the acceptability of computer-delivered treatment, especially among those with comorbid mental health and substance use problems.
The objective of our study was to assess the acceptability of a clinician-assisted computer-based (CAC) psychological treatment (delivered on DVD in a clinic-setting) for comorbid depression and alcohol or cannabis use problems relative to a therapist-delivered equivalent and a brief intervention control.
We compared treatment acceptability, in terms of treatment dropout/participation and therapeutic alliance, of therapist-delivered versus CAC psychological treatment. We randomly assigned 97 participants with current depression and problematic alcohol/cannabis use to three conditions: brief intervention (BI, one individual session delivered face to face), therapist-delivered (one initial face-to-face session plus 9 individual sessions delivered by a therapist), and CAC interventions (one initial face-to-face session plus 9 individual CAC sessions). Randomization occurred following baseline and provision of the initial session, and therapeutic alliance ratings were obtained from participants following completion of the initial session, and at sessions 5 and 10 among the therapist-delivered and CAC conditions.
Treatment retention and attendance rates were equal between therapist-delivered and CAC conditions, with 51% (34/67) completing all 10 treatment sessions. No significant differences existed between participants in therapist-delivered and CAC conditions at any point in therapy on the majority of therapeutic alliance subscales. However, relative to therapist-delivered treatment, the subscale of Client Initiative was rated significantly higher among participants allocated to the BI (F2,54 = 4.86,
Participants in a trial of CAC versus therapist-delivered treatment were equally able to engage, bond, and commit to treatment, despite comorbidity typically being associated with increased treatment dropout, problematic engagement, and complexities in treatment planning. The extent to which a client feels that they are directing therapy (Client initiative) may be an important component of change in BI and CAC intervention, especially for hazardous alcohol use.
Australian New Zealand Clinical Trials Registry ACTRN12607000437460; http://www.anzctr.org.au/trial_view.aspx?ID=82228 (Archived by WebCite at http://www.webcitation.org/5ubuRsULu)
Although mental health problems are highly prevalent, the gap between need for effective treatment and treatment received is large, particularly for counseling interventions [
Comorbidity has largely been ignored in research and policy, especially depression and alcohol/other drug (AOD) use comorbidity, and treatment services do not generally provide well for people with multiple disorders [
Brief interventions (BIs) have been widely implemented in the AOD field with a view to extending the reach of interventions, especially for alcohol problems [eg, 9]. It has been suggested that BIs are most appropriate for people with less severe drinking problems and are best combined with more intensive, longer treatments for people with moderate to severe problems [
The increased availability and use of computerized or internet-based programs as a supplement to health care is also a potential solution to accessibility problems [
We recently reported the results of the (to our knowledge) first randomized controlled trial of clinician-assisted computer-based (CAC) psychological treatment for depression and AOD use comorbidity [
A central component in the uptake and success of any treatment is acceptability to patients, particularly when translating results from clinical trials to clinical practice. This is especially relevant for different modes of treatment delivery, such as computerized therapy, which offers alternatives to traditional, face-to-face treatment. However, a recent review reported that very little attention has been paid to the acceptability of computerized psychological treatment, notably cognitive behavior therapy (CBT), compared with traditional approaches [
The present study aims to address this gap, by reporting on the acceptability of CAC CBT for comorbid depression and AOD use problems relative to an equivalent therapist-delivered CBT treatment and BI. As suggested by Kaltenthaler and colleagues [
The methods and study design have been reported in detail elsewhere [
Participants were recruited across New South Wales, Australia. Referral to the project was via a range of sources, most commonly via self-referral in response to television interviews conducted with the investigators (39/97, 40%), or newspaper articles promoting the study (53/97, 55%). A comparatively small proportion of participants were recruited via local mental health outpatient clinics (3/97, 3%) and AOD outpatient services (2/97, 2%). Following initial assessment, participants received one face-to-face session with a therapist comprising feedback, case formulation, and initial goal setting. Upon completion of this session, participants were randomized to no further treatment (BI only), nine weekly sessions of combination CBT and motivational interviewing (MI) delivered exclusively by a therapist, or nine sessions of CAC CBT/MI with weekly brief check sessions (approximately 12.5 minutes) delivered face to face by a therapist. Check-in sessions were generic in nature, comprising a check to ensure completion of the module, review of homework set for the coming week, and a mood/AOD assessment. The computerized component of CAC was DVD-based, and delivered via computers located at the study clinics. The DVD program was text-based, with interactive components including video vignettes, printable worksheets and handouts, and options for tailoring content to the participant’s stage of change or area of need. All text contained in the CAC intervention was presented by a voiceover to accommodate people with reading difficulties. Follow-up occurred 3, 6, and 12 months following baseline. Three-month (posttreatment) outcomes are reported here because of their temporal proximity to the treatment attendance and alliance indices.
The following instruments are relevant to the analyses reported below:
Data were analyzed using the Statistical Package for Social Sciences version 17.0 (SPSS Inc, Chicago, IL, USA).
Exploratory data analysis was performed on all measures relevant to the current study.
Chi-square analysis examined the proportion of treatment sessions attended (full complement vs not) for therapist-delivered and CAC condition participants. One-way analysis of variance (ANOVA) was used to examine the average attendance for the active treatment groups. For therapist-delivered and CAC condition participants, a dichotomous variable was also created to indicate whether an adequate dose of treatment had been received (yes/no). An adequate dose of treatment was considered to be attendance at 6 or more of 10 possible sessions, given that this exposed them to the majority of CBT/MI strategies included in the treatment program. Chi-square analysis was used to compare CAC and therapist-delivered condition participants on this new variable. Chi-square analysis also compared participants who completed the 3-month follow-up assessment with those who did not on gender and treatment attendance at the required number of sessions, and one-way ANOVAs examined completers and noncompleters on age, baseline levels of depression, alcohol and cannabis use, and total scores on the ARM.
Four subscales were calculated from participant responses to the ARM (Bond, Confidence, Openness, and Client Initiative). A total score was also calculated for each session (1, 5, and 10). One-way ANOVA compared scores on these subscales and total scores at each administration with treatment allocation. Change scores were created, representing the change in ARM total scores between sessions 1 and 5, sessions 1 and 10, and sessions 5 and 10, with positive scores indicating an increase in therapeutic alliance. Data were substituted with a change score of 0 when participants did not provide alliance ratings at sessions 5 and 10. Changes in ARM total scores using these variables, according to treatment allocation, were examined using one-way ANOVAs, and only for participants allocated to the therapist-delivered or CAC conditions. Power calculations were performed on the outcomes of these analyses using G*Power (Version 1.3.2, Franz Faul, Universitat Kiel, Kiel, Germany).
An average alliance total score and subscale scores were calculated for each participant, comprising the average of available ratings for each subscale or total score (n = 55). Within this dataset, Pearson correlations examined associations between average therapeutic alliance total and subscale scores and changes in depression, alcohol use, cannabis use, and hazardous use indices at the 3-month assessment relative to baseline. One-way ANOVA examined associations between alliance total scores, gender, treatment allocation, and retention. Multiple linear regression was used to predict alliance total score, using a set of predictors that included either alcohol or cannabis use variables (baseline use and stage of change), and a range of symptom (BDI-II, BHS) and treatment (allocation, adequate treatment) variables. G*Power (version 1.3.2) was used to estimate the power associated with each linear regression.
Detailed descriptions of the sample at baseline have been reported elsewhere, along with the impact of the interventions on key symptoms over a 12-month follow-up period [
As indicated in
Baseline demographics of participants in a randomized controlled trial of clinician-assisted computerized cognitive behavior therapy for coexisting depression and alcohol/other drug use problems (N = 97)
Participants | ||
Mean | SD | |
Age (years) | 35.37 | 10.21 |
Baseline levels of depression (BDI-II total score)a | 31.93 | 9.55 |
Baseline levels of alcohol use (standard drinks/day)b | 5.05 | 5.67 |
Baseline levels of cannabis use (use occasions/day)b | 10.00 | 15.06 |
Hazardous alcohol/other drug use indexc | 40.34 | 18.21 |
a Beck Depression Inventory II (BDI-II).
b Opiate Treatment Index (OTI) q score.
c Estimated day equivalents in the previous month that participants used a range of 10 drug types at harmful levels (range 0-280).
Baseline presenting symptom profiles
n | % | ||
Males:females | 45:52 | 46:54 | |
|
|||
Brief intervention - control | 30 | 31 | |
Therapist-delivered therapy | 35 | 36 | |
Clinician-assisted computer-based therapy | 32 | 33 | |
|
|||
Abstinent | 16 | 16 | |
Using - below threshold | 29 | 30 | |
Using - above threshold | 52 | 54 | |
|
|||
Abstinent | 27 | 28 | |
Using - below threshold | 1 | 1 | |
Using - above threshold | 69 | 71 | |
|
|||
Precontemplative | 27 | 28 | |
Contemplative | 34 | 35 | |
Action | 20 | 21 | |
Maintenance/abstinent | 16 | 16 | |
|
|||
Precontemplative | 10 | 10 | |
Contemplative | 39 | 40 | |
Action | 21 | 22 | |
Maintenance/abstinent | 27 | 28 |
Completion of follow-up assessments was 85% (82/97) for 3-month postbaseline, 81% at 6 months (79/97), and 85% (82/97) at 12 months. In total, 67 participants (69%) completed all phases of assessment (baseline, and 3,6, and 12 months), with no significant differences between treatment groups in follow-up participation (BI: 21/30, 70%; CAC: 23/32, 72%; therapist: 23/35, 66%; χ2
2 = 0.7,
In addition, no significant differences existed between participants who completed the 3-month follow-up assessment versus those who did not in terms of age (F1,96 = 1.25,
Mean subscale scores on the Agnew Relationship Measure (ARM) [
Subscales of the ARM | Total Score | |||||
Confidence | Client Initiative | Openness | Bond | |||
Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | Mean (SD) | ||
|
||||||
BI (17/31, 57%) | 6.13 (0.65) | 4.16 (0.90) | 5.60 (1.12) | 6.21 (0.81) | 22.10 (2.64) | |
Therapist (14/35, 40%) | 6.10 (0.80) | 3.13 (1.03) | 5.44 (1.45) | 6.29 (0.63) | 20.96 (2.73) | |
CACd (24/32, 75%) | 6.14 (0.78) | 3.52 (0.92) | 5.36 (1.58) | 6.64 (0.49) | 21.66 (2.49) | |
|
||||||
Therapist (10/35, 29%) | 6.26 (0.43) | 3.95 (0.37) | 5.58 (0.70) | 6.45 (0.37) | 22.24 (1.66) | |
CACd (20/32, 63%) | 6.10 (0.64) | 4.60 (0.62) | 5.54 (1.04) | 6.55 (0.66) | 22.78 (1.90) | |
|
||||||
Therapist (5/35, 14%) | 6.51 (0.46) | 4.05 (0.89) | 5.44 (0.99) | 6.80 (0.21) | 22.80 (1.45) | |
CACd(12/32 38%) | 6.18 (0.53) | 4.69 (0.71) | 6.03 (0.73) | 6.56 (0.49) | 23.47 (1.88) |
a Increasing scores indicate increasing levels of therapeutic alliance.
b Brief intervention (BI) – control participants did not complete these measures across all assessments given their treatment program comprised one session only.
c Rates of completion of the ARM at each session are provided as a proportion of the total number of participants allocated to each condition.
d Clinician-assisted computer-based condition (CAC) - this included therapist assistance of approximately 10 minutes per session.
As indicated in
Change scores were calculated for the change in ARM total scores between sessions 1 and 5, 1 and 10, and 5 and 10 for participants allocated to the therapist-delivered and CAC conditions. Data for participants who provided alliance ratings at session 1 but did not provide ratings at any other timepoint were substituted with a change score of 0. On average, alliance scores increased over the treatment period (mean(1 vs 5) -1.01, SD 2.48, mean(1 vs 10) 0.92, SD 1.85, mean(5 vs 10) 0.04, SD 1.21). One-way ANOVAs indicated that no significant differences existed between therapist-delivered and CAC participants in the amount of change in alliance between sessions 1 and 5 (F1,37 = 0.02,
Of the total sample, 55 (57%) provided alliance ratings following session 1, 30 provided session 5 alliance ratings, and 17 provided session 10 alliance ratings. For sessions 5 and 10 alliance ratings, this corresponded to 45% (30/67) and 25% (17/67) of eligible participants allocated to either therapist-delivered or CAC treatment (see
No significant correlations existed between any of the subscales of the ARM or the total alliance score and age, change in depression (BDI-II) scores, hopelessness (BHS) scores, and cannabis use between baseline and 3-month follow-up. This was also true for baseline levels of depression, hopelessness, and cannabis and alcohol use. A significant modest positive correlation existed between scores on the subscale of Client Initiative and change in alcohol use between baseline and 3-month follow-up (Pearson
One-way ANOVAs indicated no significant differences in alliance total score and subscale ratings and gender, stage of change for alcohol use, stage of change for cannabis use, and whether participants attended an adequate number of treatment sessions. There was a trend for treatment allocation to be associated with the subscale of Client Initiative (F2,
54 = 4.07,
Two linear regression models were used to predict the average alliance total score, using models that included either the alcohol or cannabis use variable, and a range of symptom and treatment variables. Predictor variables included baseline depression (BDI-II total score), hopelessness (BHS total score), cannabis or alcohol use at baseline (OTI score), and stage of change for alcohol/cannabis (precontemplation vs contemplation/action, or nonuse), treatment allocation, and whether adequate treatment was received (yes/no). This combination of predictors did not significantly predict alliance total scores in either the alcohol (F6,
46 = 0.60,
Given the associations between treatment allocation, change in alcohol use, and the subscale score for Client Initiative, a third linear regression model examined average Client Initiative scores, using the predictor variables of change in depression, change in hopelessness, change in alcohol use, treatment allocation, adequate treatment received, and baseline stage of change for alcohol use. This model did not significantly predict scores on the Client Initiative subscale (F6
,
46 = 0.86,
This study compared treatment acceptability, in terms of treatment dropout/participation and therapeutic alliance, of therapist-delivered versus CAC psychological treatment for comorbid depression and AOD use problems. Results indicated that both modes of treatment delivery were of equivalent acceptability to participants. This was also true for participants who received a BI. This suggests that people with comorbid depression and AOD use problems, despite the engagement, retention, and treatment difficulties characteristic of this population, can develop strong attachment with a computer-delivered treatment program and commitment to complete an adequate dose of treatment with minimal therapist input. These results are discussed in detail below.
All participants were randomly assigned to therapist-delivered versus CAC treatment following one face-to-face session. Take-up rates of both modes of treatment were high following randomization, with 91% (32/35) of therapist-delivered and 97% (31/32) of CAC treatment participants returning for at least one session. Over the 10 sessions of active treatment, no statistically significant differences were evident between the treatment groups in patterns of treatment attendance. Therefore, according to this index of acceptability, it is reasonable to suggest that people in the CAC treatment found this mode of delivery as acceptable as a therapist-delivered alternative. In a recent review of the acceptability of computerized CBT for depression [
Results relating to the second criterion of acceptability, therapeutic alliance, also suggested equivalence in outcomes between therapist-delivered and CAC treatments, and, for session 1, a BI. Participants rated therapeutic bond, confidence in therapy, ability to direct therapy, and client openness highly across the treatment conditions at sessions 1 (all conditions), 5, and 10 (therapist-delivered and CAC treatments). It is of note that Client Initiative was rated significantly higher by participants in the CAC condition at session 5, relative to the therapist-delivered condition. Although this difference had disappeared by session 10, it suggests increased empowerment and enhanced problem-solving skills potentially associated with the “self-help” nature of computer-based treatment. As a similar result regarding Client Initiative was obtained for the BI relative to the therapist-delivered alternative after session 1, similar alliance mechanisms may be operating in the BI and CAC conditions among this comorbid group. Over the course of treatment, total alliance scores increased by 2 points from session 1 to session 10, with no significant differences evident between the therapist-delivered and CAC treatment groups. In addition, therapeutic alliance scores (total and subscale scores) across all time points were not predicted by treatment allocation, nor by any of the models tested in the regression analysis.
No previous study has reported on therapeutic alliance among people completing therapist-delivered versus CAC treatments for depression and AOD use problems; however, studies of computerized CBT for other mental health conditions have generally reported patient satisfaction and acceptability of this mode of delivery [
The real-world implications of these results are potentially important. Namely, a group of people with moderate and severe levels of comorbid depression and AOD use problems, who are challenging to engage and retain, and are regarded as complicated to treat effectively [
Early alliance ratings (session 5 or earlier) have generally demonstrated higher predictive value, in terms of symptom reduction and other posttreatment outcomes, than later-therapy alliance and/or average alliance [
There are several limitations to this study, not the least of which is the small sample size and participant attrition in relation to therapist alliance ratings. In substituting data for participants who did not complete the session 5 and session 10 ARM ratings, we assumed no change, when alliance may have deteriorated. This may have inflated the improvement observed in therapeutic alliance over the treatment period reported in relation to
No previous research has examined the acceptability and therapeutic alliance of CAC therapy among a group with comorbid depression and AOD use relative to a BI or therapist equivalent, nor with a sample reporting severe levels of depression at baseline and concurrent heavy use of alcohol or cannabis. The results indicate that people with this comorbidity find CAC treatment as acceptable, in terms of treatment dropout and therapeutic alliance, as an equivalent therapist-delivered treatment program. This robust finding was demonstrated across a range of potentially confounding demographic and symptom domains. Rates of dropout in both treatment modalities were equivalent to other treatment trials among people with depression, and among those participating in trials of CBT, despite the study population having current and severe comorbidity and being stereotypically difficult to attract, retain, and treat effectively.
The extent to which client characteristics and alliance may work together to moderate posttreatment outcomes still needs to be determined. Symptom and functioning outcomes of CAC versus therapist-delivered treatment have been reported elsewhere [
The promising results regarding the acceptability of CAC treatment to a complex comorbid group are important, considering that the computer-delivered intervention used an average of 12.5 minutes face-to-face clinician time per session compared with approximately 1 hour of face-to-face therapy among the therapist-delivered equivalents. In Australia, 67% of people with mental health problems do not access treatment for their conditions [
The authors wish to acknowledge the involvement of the study participants without whom this research would not be possible. The study was funded in part by the Alcohol-related Medical Research Scheme (Australian Brewer’s Foundation), and a bequest from Ms Jennie Thomas on behalf of her late husband Philip Emlyn Thomas via the University of Newcastle, Australia. In addition, a National Health and Medical Research Council (NHMRC) public health postgraduate scholarship supported the primary author. The research team remained independent from the funding bodies.
This study was carried out in accordance with the National Health and Medical Research Council of Australia’s Statement of Ethical Conduct of Research among Human Participants. Ethics approval was gained from the relevant Human Research Ethics Committees (HAREC Approval No: 02/03/13/3.16, HREC Approval No: H 307 0502).
None declared
analysis of variance
alcohol/other drug
Agnew-Davies Relationship Measure
Beck Depression Inventory II
Beck Hopelessness Scale
brief intervention
clinician-assisted computer-based
cognitive behavior therapy
motivational interviewing
Opiate Treatment Index