This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Depression is a common mental disorder with a high social burden and significant impact on suicidality and quality of life. Treatment is often limited to drug therapies because of long waiting times to see psychological therapists face to face, despite several guidelines recommending that psychological treatments should be first-line interventions for mild to moderate depression.
We aimed to evaluate, among patients on a waitlist to receive secondary mental health care services for depression, how effective coach-guided web-based therapy (
We conducted a randomized controlled trial with 2 parallel arms and a process evaluation, which included interviews with study participants. Participants assigned to the intervention group received 12 weeks of web-based therapy guided by a coach who had a background in social work. Patients in the control group receive a leaflet of mental health resources they could access. The primary outcome measure was a change in depression scores, as measured by the Patient-Health Questionnaire (PHQ-9).
A total of 95 participants were enrolled (intervention, n=47; control, n=48). The mean change in PHQ-9 scores from baseline to week 12 was −3.6 (SD 6.6) in the intervention group and −3.1 (SD 6.2) in the control group, which was not a statistically significant difference with a two-sided alpha of .05 (t91=−0.37;
The results demonstrate that the use of guided web-based therapy for the treatment of depression is not more effective than information-only waitlist control. However, it showed that the coach has the potential to increase adherence and engagement with web-based depression treatment protocols. Further research is needed on what makes the coach effective.
ClinicalTrials.gov: NCT02423733; https://clinicaltrials.gov/ct2/show/NCT02423733
Depression is a common mental disorder [
There is evidence that web-based therapies can reduce the symptoms of depression [
The delivery of web-based therapy as a treatment for depression can be performed in one of two ways: through the use of supports to assist patients through the web-based therapy (guided model) or through the self-help use of computerized treatment (unguided model). In the guided model, patients are provided support as they progress through web-based therapy. In some cases, highly trained clinicians have been used to fill this role [
Several systematic reviews have demonstrated that internet-based interventions for depression have effect sizes that are comparable with face-to-face interventions, whereas unguided interventions have smaller effect sizes [
Web-based interventions for depression struggle with engagement and adherence to treatment protocols. For example, in the
We have previously reported on an RCT of
The design of the trial was an RCT with two parallel groups. This trial has been reported according to the Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth (CONSORT-EHEALTH;
Potential participants were patients referred to the Royal Ottawa Mental Health Centre (ROMHC, Ottawa, Canada) with symptoms of depression or dysthymia who were on a waiting list for treatment in the following psychiatric programs: Mood and Anxiety, Geriatric Psychiatry and Youth Psychiatry. The ROMHC has 284 inpatient beds and acts as a specialized mental health facility for residents of communities across Eastern Ontario. These programs are aimed at treating people with complex and serious mental illnesses that are often resistant to treatment. Patients are referred to treatment in these programs directly by their family physicians. The first appointment in the programs is with a psychiatrist who then decides on a treatment plan with the patient. At the time of this clinical trial, the waitlist for the Mood and Anxiety Program was between 9 months and 1 year.
Eligibility criteria for participation in the trial are outlined in
Inclusion criteria:
16 years of age or older.
Referred and triaged to the Mood and Anxiety, Youth Psychiatry, or Geriatric Psychiatry Program at the Royal Ottawa Mental Health Centre for any depressive symptoms.
Willing to attend electronic therapy sessions for up to 12 weeks.
Able and willing to provide informed consent.
Willing to be randomized.
Willing to comply with all study procedures.
Exclusion criteria:
Is unable to read or write in English.
Does not have an Ontario Health Insurance Plan number.
Has cognitive impairments that render them unable to use a computer.
There is another participant enrolled in the study who lives at their address.
As described in
Breakdown of participant progress through The Journal.
Lessons | Description | Tasks to be completed by participants | |||
Positivity module | Learn the importance of staying positive and planning regular activities that they enjoy |
Watch video on staying positive; Select 2 enjoyable activities; Select dates to complete activities. |
|||
|
|||||
|
Eating right | Explore the link between diet and mood. |
Watch video on eating right; Browse and select a healthy recipe; Create a shopping plan. |
||
|
Getting active | Review benefits of being active on mood. |
Watch video on getting active; Pick 2 activities to complete; Make a plan for getting active. |
||
|
Learning to relax | Highlights the importance of stress management. |
Watch video on learning to relax; Practice relaxation and breathing; Make a plan for relaxing exercises. |
||
|
Sleeping better | Discuss the importance of good sleep habits to mood. |
Watch video on sleeping better; Set a nighttime routine; Set a morning routine; Keep a sleep diary; Make a plan to practice sleep hygiene. |
||
|
|||||
|
Identify problems | Learn how depression impacts problem-solving abilities. |
Watch video on identifying problems; Create a problem list; Pick a problem to work on; Define the problem; Make a plan to create a problem list and statement. |
||
|
Find solutions | Explore how to use both logical and creative parts of the brain to brainstorm problem solutions. |
Watch video on brainstorming solutions; Create a solutions list; Select a solution to implement; Evaluate solutions; Make a plan to list and evaluate solutions. |
||
|
Create a plan | Review how to create SMARTb plan. |
Watch video on brainstorming solutions; Review the selected solution to make sure it is SMART; Write a detailed step-by-step plan; Review plan. |
||
|
Review your plan | Highlights the importance of assessing progress and updating the SMART plan. |
Watch video on reviewing the SMART plan; Review progress on plan; Revise plan as needed; Complete self-test. |
aParticipants are only required to complete 1 of the 4 lifestyle lessons.
bSMART: specific, measurable, achievable, relevant, time-bound.
Both groups received usual care while on the waitlist, which included management by a family physician and use of community resources, such as access to distress center lines and counseling services. Once participants are called off the waitlist, they receive an initial appointment with a psychiatrist, at which point adjustments are made to the patient’s care plan. They are then referred back to their family physician, or they receive further psychiatric treatment (eg, regular care from a psychiatrist, social worker, nurse, occupational therapist, and so on).
Following the consent appointment, participants were randomized to one of two treatment groups. In addition to usual care, participants assigned to the control group received an information leaflet with web-based resources, including
For those participants assigned to the intervention group, the intervention consisted of the following:
An information leaflet of web-based depression resources.
An invitation to use
12-weekly telephone coaching sessions with a coach (SL), who had a guideline script for each coaching session, reinforced the topic of each lesson in
Text message or email contact between appointments, as per the participant’s preference.
The primary outcome measure was the Patient Health Questionnaire (PHQ-9) [
Suicidal thoughts were assessed by question 9 of the PHQ-9, in which respondents were asked
Health-related quality of life was assessed using the EuroQol-5 dimension (3 levels) questionnaire (EQ-5D-3L). This is a 5-item questionnaire that assesses health-related quality of life, including mobility, self-care, ability to participate in one’s usual activities, pain or discomfort, and anxiety or depression. The EQ-5D-3L asks participants to assess their health-related quality of life on a 3-point scale from no dysfunction to extreme dysfunction, with the following response categories:
Level 1: indicating no problem.
Level 2: indicating some problems.
Level 3: indicating extreme problems.
The EQ-5D-3L is then able to define a unique health state based on the responses to each of the 5 dimensions of health described above. Respondents fall into 1 of 243 different health states, depending on their responses to the questionnaire [
Service use was measured using data extracted from participants’ electronic medical records (EMR), including, time to first consultation appointment at the ROMHC and the total number of outpatient mental health follow-up appointments after the first consultation completed at the ROMHC. These measures were administered as shown in
Baseline assessments were administered in-person following the consent appointment, and all other time point assessments were conducted by telephone either by the coach (intervention group) or a research assistant (control group). Patients who missed appointments or were lost to follow-up were also sent questionnaires by mail. Service use at the ROMHC was obtained from the EMR of each participant. Five participants in the control group were excluded ad hoc to prevent confounding as they accessed
Outcome measures and timing of assessments.
Variable | Outcome measure | Time point | |
|
|||
|
Depressive symptoms | PHQ-9a | Baseline, week 2, week 6, week 12 |
|
|||
|
Suicidal thoughts | PHQ-9 Q9b | Baseline, week 2, week 6, week 12 |
|
Health-related quality of life | EQ-5D-3Lc, EQ-5D-VASd | Baseline, week 6, week 12 |
|
Health service use | ROMHC EMRe | One year following the initial consultation appointment at the ROMHC |
aPHQ-9: Patient Health Questionnaire.
bPHQ-9 Q9: Patient Health Questionnaire question 9.
cEQ-5D-3L: EuroQol 5 dimensions (3 levels) questionnaire.
dEQ-5D-VAS: EuroQol 5 dimensions visual analogue scale.
eROMHC EMR: Royal Ottawa Mental Health Center electronic medical record.
Based on the previous studies that used the PHQ-9 as their primary outcome measure, we expected the mean pretreatment score to be 17.0 (SD 4.0). To detect a difference in PHQ-9 scores between the two groups of at least 3 points, an established minimal clinically important difference [
Randomization was completed by the Ottawa Methods Centre at the Ottawa Hospital Research Institute, with allocations kept in sequential sealed envelopes at the study base. Participants were randomized in a 1:1 allocation, and there were no restrictions. After providing consent, participants were randomized by a research assistant according to the allocation in the sealed envelopes.
Owing to the nature of the intervention, neither participants nor study staff were blinded to the treatment allocation. Outcome assessments were collected by delegated study staff who were not blinded to the treatment allocation.
Group differences in demographic and pretreatment measures were analyzed using independent samples
Differences in suicidal thoughts between the two groups were assessed using an independent samples
As per the recommendations outlined in the Medical Research Council’s guidelines for the assessment of complex interventions [
Process evaluation outcome measures.
Evaluation critierion and outcome measure | Description | ||
|
|||
|
Facilitators of and barriers to study completion |
Qualitative interviews with participants |
|
|
|||
|
Reach |
Total number of participants reached Comparison of sample to Ontarian and Canadian populations. |
|
|
Fidelity |
Mean length of weekly coaching calls Mean number of contacts with the coach Mean number of weekly coaching sessions completed Mean number of lessons in |
|
|
Dose |
Total number of participants to complete 6 lessons in |
|
|
|||
|
Role of the coach |
Qualitative interviews with participants |
aAll usage data from
Individual interviews were conducted by two female University of Ottawa Department of Psychiatry residents (JK and ZS) who were independent of the research team that conducted the RCT. Following transcription, two independent coders (JK and KG) analyzed the material using a thematic, grounded theory approach. The coding took place in two stages, with coders meeting during the first stage to discuss emergent codes, reconcile definitions, and compare coding rationales. The two coders had not been involved in delivering the treatment or any of the previous study contacts.
The study received approval from the Royal Research Ethics Board (Protocol 2014001). All participants provided informed consent before participation in both the RCT and qualitative interviews.
Recruitment for this study took place over 11 months, from May 2015 to April 2016, and a total of 1316 patients were preliminarily screened for eligibility by examining their referral documentation (
Of the 45.9% (605/1316) of patients who could not be contacted to complete a full eligibility screen, 25.4% (154/605) provided consent to be approached about research studies but did not reply to telephone or mail invitations; 63.1% (382/605) were invited to discuss the study by mail with no response, and 11.3% (69/606) had incomplete mailing information listed on their referral documents.
Of the 40.4% (532/1316) of patients who could be contacted but were ineligible to participate, 84.9% (452/532) were referred for a reason other than depressive symptoms; 5.6% (30/532) had cognitive impairments rendering them unable to use a computer; 4.5% (24/532) were no longer on the waitlist at the time of screening; 2.6% (14/532) were under 16 years of age at the time of screening; 1.12% (6/532) were unable to read and write in English; and, 1.12% (6/532) did not have a valid Ontario Health Insurance Plan number.
Of the 13.6% (179/1316) of patients who were eligible to participate, 46.9% (84/179) declined to participate, and 53.1% (95/179) consented to participate in the study.
The majority of participants enrolled in the study were recruited from the Mood and Anxiety Outpatient Clinic (n=92), and 3 were recruited from the Geriatric Psychiatry Program at the ROMHC. No participants were enrolled from the Youth Psychiatry Program.
Participants in the intervention group completed a mean number of 5.0 (SD 2.3) lessons in
Consolidated Standards of Reporting Trials flow and attrition diagram. CONSORT: Consolidated Standards of Reporting Trials.
Reasons for nonparticipation (n=84).
Reason for Nonparticipation | Value, n (%) |
Did not attend baseline intake appointment | 27 (32) |
Did not have computer/internet at home—not interested in going to public library or community center | 16 (19) |
No reason provided | 13 (15) |
Interested in participating in a different study also recruiting from the mood and anxiety program | 6 (7) |
Too overwhelming | 4 (5) |
Would prefer to wait for appointment with psychiatrist | 3 (4) |
Not interested in participating in research at the Royal Ottawa Hospital | 3 (4) |
Did not feel that the study would benefit them | 2 (4) |
Not interested in weekly contact | 2 (2) |
No time | 2 (2) |
Family circumstances | 1 (1) |
Participating in another research study | 1 (1) |
Interested in medication change or recommendations | 1 (1) |
Migraines due to computer use | 1 (1) |
Moving out of province | 1 (1) |
Interested only in face-to-face therapy | 1 (1) |
Sample demographic characteristics.
Demographic characteristic | Total (n=95) | Control group (n=48) | Intervention group (n=47) | |
|
||||
|
Male | 28 (30) | 20 (42) | 8 (17) |
|
Female | 66 (70) | 28 (58) | 38 (83)b |
Age (years), mean (SD) | 44.2 (12.9) | 44.8 (13.7) | 43.5 (12.1) | |
|
||||
|
First Nations | 1 (1) | 0 (0) | 1 (2) |
|
Inuk | 0 (0) | 0 (0) | 0 (0) |
|
Métis | 3 (3) | 3 (6) | 0 (0) |
|
Asian | 3 (3) | 2 (4) | 1 (2) |
|
African, Caribbean, or Black | 0 (0) | 0 (0) | 0 (0) |
|
White | 82 (87) | 42 (88) | 40 (87) |
|
Other | 5 (5) | 1 (2) | 4 (9) |
|
||||
|
Single | 33 (35) | 11 (23) | 22 (47) |
|
Common law | 6 (6) | 4 (8) | 2 (4) |
|
Married | 35 (37) | 18 (38) | 17 (36) |
|
Separated | 4 (4) | 3 (6) | 1 (2) |
|
Divorced | 16 (17) | 11 (23) | 5 (11) |
|
Widowed | 1 (1) | 1 (2) | 0 (0) |
|
||||
|
High School | 10 (11) | 7 (15) | 3 (6) |
|
College | 45 (47) | 23 (48) | 22 (47) |
|
University—undergraduate | 26 (27) | 10 (21) | 16 (34) |
|
University—master’s | 10 (11) | 5 (10) | 5 (11) |
|
University—doctorate | 4 (4) | 3 (6) | 1 (2) |
|
||||
|
Full-time | 23 (24) | 11 (23) | 12 (26) |
|
Part-time | 8 (8) | 2 (4) | 6 (13) |
|
Short-term disability | 5 (5) | 2 (4) | 3 (6) |
|
Long-term disability | 31 (33) | 18 (38) | 13 (28) |
|
Self-employed | 7 (7) | 3 (6) | 4 (8) |
|
Retired | 10 (11) | 5 (10) | 5 (11) |
|
Unemployed | 11 (12) | 7 (15) | 4 (8) |
an=1 transgender participant removed from the analysis.
b
The mean PHQ-9 score was lower in the intervention group than in the control group at all study time points (
At 12 weeks, the mean PHQ-9 score in the intervention group was 11.3 (SD 6.4) and was 12.4 (SD 6.4) in the control group (t86=0.76;
Excluding the 10% (5/48) of participants in the control group who also accessed
Mean and median scores on the Patient-Health Questionnaire-9 (n=95).
Study time point | Mean (SD) | Median | Missing values, n (%) | Mean difference (95% CI) | Independent samples |
||
|
|
|
|
||||
|
Control (n=48) | 15.4 (5.4) | 16.5 | 0 (0) |
|
|
|
|
Intervention (n=47) | 14.9 (6.0) | 16.0 | 0 (0) |
|
|
|
|
|
|
|
||||
|
Control (n=46) | 12.6 (5.5) | 11.5 | 2 (4) |
|
|
|
|
Intervention (n=45) | 11.5 (5.9) | 12.0 | 2 (4) |
|
|
|
|
|
|
|
||||
|
Control (n=46) | 12.4 (6.2) | 12.0 | 2 (4) |
|
|
|
|
Intervention (n=45) | 10.7 (6.1) | 10.0 | 2 (4) |
|
|
|
|
|
|
|
||||
|
Control (n=46) | 12.4 (6.4) | 11.5 | 2 (4) |
|
|
|
|
Intervention (n=47) | 11.3 (6.4) | 10 | 0 (0) |
|
|
|
The repeated measures modeling found that scores on the PHQ-9 differed significantly by study time point, irrespective of group allocation (
After 12 weeks of treatment, participants in the intervention group reported higher health-related quality of life, with mean index scores on the EQ-5D-3L of 0.7 (SD 0.7) for the intervention group and 0.6 (SD 0.5) for the control group (t86=−2.30;
At 12 weeks, of the 47 participants in the intervention group, 66% (31/47) reported no suicidality, 28% (13/47) reported suicidality several days in the preceding 12 weeks, 2% (1/47) reported suicidality nearly half the days, and 4% (2/47) reported suicidality nearly every day. In comparison, of the 46 participants in the control group, 69% (33/46) reported no suicidality, 17% (8/46) reported suicidality several days, 6% (3/46) reported suicidality more than half the days, and 4% (2/46) reported suicidality nearly every day (
Mean and median scores on the Patient-Health Questionnaire -9, Question 9 (n=95).
Study time point | Control (n=48), n (%) | Intervention (n=47), n (%) | Chi-square ( |
||||||
|
|
|
|||||||
|
Not at all | 24 (50) | 23 (49) |
|
|
||||
|
Several days | 16 (33) | 18 (38) |
|
|
||||
|
More than half the days | 5 (11) | 4 (9) |
|
|
||||
|
Nearly everyday | 3 (6) | 2 (4) |
|
|
||||
|
Total | 48 (100.0) | 47 (100.0) |
|
|
||||
|
|
|
|||||||
|
Not at all | 30 (63) | 34 (72) |
|
|
||||
|
Several days | 12 (25) | 8 (17) |
|
|
||||
|
More than half the days | 2 (4) | 2 (4) |
|
|
||||
|
Nearly everyday | 2 (4) | 1 (2) |
|
|
||||
|
Total | 46 (96) | 45 (95) |
|
|
||||
|
|
|
|||||||
|
Not at all | 27 (56) | 32 (68) |
|
|
||||
|
Several days | 11 (23) | 11 (23) |
|
|
||||
|
More than half the days | 5 (10) | 2 (4) |
|
|
||||
|
Nearly everyday | 3 (6) | 0 (0.0) |
|
|
||||
|
Total | 46 (95) | 45 (95) |
|
|
||||
|
|
|
|||||||
|
Not at all | 33 (69) | 31 (66) |
|
|
||||
|
Several days | 8 (17) | 13 (28) |
|
|
||||
|
More than half the days | 3 (6) | 1 (2) |
|
|
||||
|
Nearly everyday | 2 (4) | 2 (4) |
|
|
||||
|
Total | 46 (96) | 47 (100.0) |
|
|
Participants in both groups received similar levels of face-to-face follow-up care, with 25.0% of participants in the intervention group receiving any follow-up in the next 12 months at the hospital after their initial outpatient appointment with a psychiatrist compared with 21.3% of the control group (Χ21=0.7;
Hospital service use.
Health service use indicator | Control | Intervention | Tests of association | |
Number of days from referral to first appointmenta, mean (SD) | 213.6 (54.6) | 219.3 (57.0) | t69=−0.43 |
.67 |
Number of people who received outpatient follow-up by a nonpsychiatrist after initial assessment by a psychiatrist, n (%) | 13/44 (30) | 12/46 (26) | Χ21=0.1 | .71 |
Number of people who received outpatient follow-up by a psychiatrist after their initial consultation, n (%) | 20/44 (42) | 18/46 (38) | Χ21=0.3 | .54 |
Number of outpatient follow-up appointments with a psychiatrist in the year after the initial consultationb, mean (SD) | 2.3 (3.1) | 2.5 (4.6) | t88=−0.30 | .76 |
Number of outpatient follow-up appointments with all disciplines in the year after initial consultationc, mean (SD) | 3.6 (5.6) | 4.8 (8.5) | M-Wd U=255.5, Z=−0.11 | .91 |
aControl group n=34; Intervention group n=37.
bControl group n=34; Intervention group n=37.
cControl group n=34; Intervention group n=37.
dM-W: Mann-Whitney.
As reported above, only 13.4% (95/711) of potentially eligible patients (eg, those who could be contacted and were on the waitlist for mental health treatment) agreed to participate in the trial. Of particular note is that 8.9% (16/179) eligible participants declined to participate because they did not have computer or internet access at home. This is comparable with 14% of the Canadian population [
Analysis of trial records demonstrated that the study intervention was implemented as intended in the study protocol. Participants in the intervention group completed a mean of 5.0 (SD 2.3) lessons in
Relationship between fidelity measures and Patient-Health Questionnaire scores at 12 weeks (n=47).
Fidelity Measures | Mean (SD) | Median | Pearson’s correlation with PHQ-9a scores at week 12, (n=47) | ||||||
|
|
|
|
||||||
Total lessons completed in |
5.0 (2.3) | 6.0 | −0.436 | .002 | |||||
Total sessions with the coach | 8.8 (3.1) | 10.0 | −0.435 | .002 | |||||
Average length of coaching calls (min) | 30.8 (12.9) | 27.7 | −0.360 | .01 | |||||
|
|||||||||
|
Telephone | 13.1 (4.0) | 13.0 | 0.061 | .68 | ||||
|
2.0 (2.5) | 1.0 | −0.163 | .27 | |||||
|
Text message | 25.7(15.4) | 26.0 | −0.073 | .62 |
aPHQ-9: Patient Health Questionnaire.
For those participants in the intervention group who received a full
Similarly, by week 12 in the intervention group, 51% (24/47) of participants had a clinically important reduction in depressive symptoms (PHQ-9 score of 9 or less or a 50% or more improvement in scores) compared with 37% (17/46) of participants in the control group. However, this difference was not statistically significant (Χ21=1.8;
A total of 15 qualitative interviews were conducted (10 intervention, 5 control) with 4 male (2 intervention group, 2 control group), 10 female (7 intervention group, 3 control group), and 1 transgender person (intervention group). Interviews took place at the ROMHC at a time that was mutually convenient for both participants and interviewers. Themes that arose from the interviewers were categorized in terms of whether there were positive facilitators of or negative barriers to engagement with the study intervention.
Participants identified Sir John Kirwan’s relatability as a positive facilitator through his sharing of his personal experience with depression. For instance, a participant commented on the videos that begin each lesson in
…I just like the way that they interacted with each other and how casual it was, so it didn’t feel like I was doing homework, and it didn’t feel like I was doing medical stuff…It was just, watching two people sitting on a bench in a park and they were talking, and I really liked how the narrator came right out and said, you know, when I was depressed, you know, I couldn’t get out of bed, or I, I didn’t want to take a shower, it was, like, wow somebody else feels the way I do. So, it helped me to feel like I wasn’t all, all alone. And he was, he seemed to be very honest about his experience.
Similarly, participants expressed an appreciation for the structure and layout of
Interviewer: …what did you like about the problem-solving lesson?
Participant: … it was in a new, new approaches. It gave me a new technique or I found a new technique in there in terms of outlining problems, and thinking about them and, that I hadn’t.
Participants also reported frustrations with some aspects of
Participant: Yeah, lifestyle one…was little bit of a challenge.
Interviewer: Yeah? In what way?
Participant: … just trying to find motivation…it’s just hard to change some things about lifestyle, but it came with time”
Similarly, a participant in the control group commented on how some depressive symptoms, such as difficulty with concentration, may act as barriers to engagement with treatment:
Interviewer: … Were there any barriers or challenges to using the brochure?
Participant: …not really because it’s short and sweet…my only barrier was time, you know, time and, and energy. Because at the time, and it was almost an opportune time that I was doing this because I was low at the time… so when you are depressed the last thing you want to do it read, so you can’t even. And that’s why I was saying that if this was little bullet chunks, you know, you don’t have to read it so much, cause I can’t read a book when I’m down. Like I start to read a paragraph and I’m like, no I have to close it.
Numerous participants in the intervention group also identified the coach as key to their success with the program, highlighting the importance of accountability:
Interviewer: …what would you say made the biggest difference to your participation in the study? ...was it the journal, was it the coach?
Participant: It was the coach, oh my god. I would not have stayed with that journal unless [SL] was calling me… cause then I wouldn’t really have anybody to be like, hey did you do that thing?
Similarly, another participant questioned whether they would have been as successful in completing the program had they not had someone following-up on their goals:
…if it was by myself, I might have just kept postponing it or not doing it or, so I found it quite helpful. She's a pretty great coach.
Similarly, another participant highlighted the importance of continuity between the lessons in
I like the conversations the following week with the homework that we did. I liked that it was continuous.
Accountability also emerged as important for participants in the control group. For instance, 1 participant who accessed
…you guys can follow-up and see if they’ re.. If people are using it or not…just to say, somebody is following me, I should probably do a little bit more of this...
This was echoed by another participant in the control group who struggled with the limited follow-up they received from the research team:
taking not even five minutes to say “how are you doing right now?”… So in a way it gauges their, their level of health, of mental health at the time, and if they do appear to need resources then [the Research Assistant] could say “look I’m not a doctor, I’m not a psychiatrist, but do you remember the flyer that I gave you, on it there was this thing that might be helpful for you right now, you might want to give it a thought”… so, that’s just like extra, like value added for resourcefulness.
Among patients on a waiting list referred to secondary care for depressive symptoms, a trial of usual care plus coach-facilitated web-based therapy compared with usual care plus information-only waitlist control group found no statistically significant differences in mean depression scores after 12 weeks. However, participants who completed 6 or more lessons in
The major strength of this study is that it is the first study of coach-assisted web-based therapy in a secondary care setting in Canada. This study demonstrated that it is possible to incorporate the use of a coach within this clinical setting. However, due to limitations imposed by institutional policies, study staff were restricted in their ability to recruit participants, with only 12% (95/784) of potentially eligible patients agreeing to participate in the trial. It is reasonable to expect greater reach if this intervention was rolled out by a clinical service that could approach people directly at the time of referral.
There was also a possibility of contamination between the arms of the trial. Given that
Some participants who were interviewed as part of the process evaluation expressed issues with recall given that the interviews took place 6 months after study completion. It is possible that interviews conducted closer to study completion would have yielded richer qualitative data.
A key contribution of this study is that participant engagement with and adherence to the intervention was much higher than that reported in previous work, with 62% (29/47) of participants in the intervention group receiving a full
The previous trial of a guided version of
Web-based therapies are often promoted as a way to address long waiting lists for mental health care services. However, in this study, even though participants in the intervention group with more exposure to the study intervention were more likely to experience a significant reduction in symptoms, this had little impact on subsequent service use. Web-based therapies are part of a complex socio-technological system and, as such, cannot exist in a vacuum. To achieve improvement in patient outcomes, they must be integrated into a larger system of care. This could be achieved with a more explicit stepped care system supported by web-based therapies, with a more flexible response from providers based on patient need.
This study provides limited support for the potential use of web-based therapies within a stepped-care approach to the treatment of depression. However, an RCT is needed to determine the effectiveness of such an approach. The impact of providing digital services to those in greatest need, who are also the least likely to have access to high-speed connections, also needs to be taken into account. Future research on internet-based psychotherapy for depression needs to include the system of care in which it is used. This can be done through implementation science tools, which not only evaluate the effectiveness of web-based therapies but also the factors central to their uptake, such as reach, adoption, and sustainability.
The results of this study demonstrated that the use of guided web-based therapy for the treatment of depression is not more effective than information-only waitlist control. However, it showed that coach-guided web-based therapy has the potential to increase adherence and engagement with depression treatment protocols. Greater adherence resulted in greater effectiveness. More research is needed on the human component of coaching in conjunction with web-based therapy to examine what factors lead to greater adherence. Researchers also need to consider when and how web-based therapies should be integrated into existing clinical pathways.
CONSORT EHEALTH Checklist V1.6.
Supplemental file – Screenshots from The Journal.
Mean and median scores on the EuroQol 5 dimension (3 levels) visual analog scale (n=94).
Cognitive behavioral therapy
electronic medical record
EuroQol 5 Dimensions (3 levels)
EuroQol 5 dimensions visual analogue scale
mean difference
National Institute for Health and Care Excellence
Patient Health Questionnaire question 9
Patient Health Questionnaire
randomized controlled trial.
Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy
Royal Ottawa Mental Health Centre.
This work was supported by a Canadian Institutes for Health Research Catalyst grant (funding reference number: 134160) and the Royal’s University Medical Research Fund. The funders had no role in the review or approval of the manuscript for publication.
The research team would like to recognize the following psychiatric programs at the ROMHC: Mood and Anxiety, Geriatric Psychiatry, and Youth Psychiatry, as well as the service users who participated in our study.
SH conceived of the study and was the grant holder. SM took the lead in drafting of the manuscript, completed participant recruitment, and data collection activities. DJC conducted all statistical analyses. SL completed participant follow-up and data collection activities. JK and ZS completed the process evaluation of qualitative interviews. JK and KG conducted qualitative analysis of the process. All authors reviewed and approved the final manuscript.
None declared.