Background: Depression is a common mental disorder with a high social burden and significant impact on suicidality and quality of life. Treatment is often limited to drug therapies because of long waiting times to see psychological therapists face to face, despite several guidelines recommending that psychological treatments should be first-line interventions for mild to moderate depression.
Objective: We aimed to evaluate, among patients on a waitlist to receive secondary mental health care services for depression, how effective coach-guided web-based therapy (The Journal) is, compared with an information-only waitlist control group, in reducing depression symptoms after 12 weeks.
Methods: We conducted a randomized controlled trial with 2 parallel arms and a process evaluation, which included interviews with study participants. Participants assigned to the intervention group received 12 weeks of web-based therapy guided by a coach who had a background in social work. Patients in the control group receive a leaflet of mental health resources they could access. The primary outcome measure was a change in depression scores, as measured by the Patient-Health Questionnaire (PHQ-9).
Results: A total of 95 participants were enrolled (intervention, n=47; control, n=48). The mean change in PHQ-9 scores from baseline to week 12 was −3.6 (SD 6.6) in the intervention group and −3.1 (SD 6.2) in the control group, which was not a statistically significant difference with a two-sided alpha of .05 (t91=−0.37; P=.72, 95% CI −3.1 to 2.2). At 12 weeks, participants in the intervention group reported higher health-related quality of life (mean EuroQol 5 dimensions visual analogue scale [EQ-5D-VAS] score 66.8, SD 18.0) compared with the control group (mean EQ-5D VAS score 55.9, SD 19.2; t84=−2.73; P=.01). There were no statistically significant differences between the two groups in health service use following their initial consultation with a psychiatrist. The process evaluation showed that participants in the intervention group completed a mean of 5.0 (SD 2.3) lessons in The Journal and 8.8 (SD 3.1) sessions with the coach. Most participants (29/47, 62%) in the intervention group who completed the full dose of the intervention, by finishing 6 or more lessons in The Journal, were more likely to have a clinically important reduction in depressive symptoms at 12 weeks compared with the control group (Χ21=6.3; P=.01, Φ=0.37). Participants who completed the interviews reported that the role played by the coach was a major factor in adherence to the study intervention.
Conclusions: The results demonstrate that the use of guided web-based therapy for the treatment of depression is not more effective than information-only waitlist control. However, it showed that the coach has the potential to increase adherence and engagement with web-based depression treatment protocols. Further research is needed on what makes the coach effective.
Trial Registration: ClinicalTrials.gov: NCT02423733; https://clinicaltrials.gov/ct2/show/NCT02423733
Depression is a common mental disorder  with a high social burden [ ] and significant impact on suicidality [ ] and quality of life. Most treatment for depression occurs in primary care. In secondary care, treatment is often limited to drug therapies, in part, due to long waiting lists to see psychological therapists face-to-face. This is contrary to recommendations about the importance of nondrug therapies by the National Institute for Health and Care Excellence (NICE), based in the United Kingdom, and other institutions [ ]. In secondary and tertiary mental health care centers in Ontario, at the time of this trial, the waiting time to be seen by a psychiatrist for depression was between 9 months and 1 year.
Use of Electronic Therapies in Treating Depression
There is evidence that web-based therapies can reduce the symptoms of depression . Randomized controlled trials (RCTs) have demonstrated the effectiveness of web-based cognitive behavior therapy (CBT) [ , ], problem-solving therapy [ ], interpersonal therapy [ ], and psychodynamic therapy. Computerized CBT is also recommended in the NICE guidelines for the treatment of mild to moderate depression [ ]. However, most of the RCTs of web-based therapies have been conducted in community samples, often recruited from the internet. These populations are self-selecting and, although their scores on depression rating scales may be comparable with clinical populations, they often differ in terms of comorbidity, duration of symptoms, and impact on daily functioning.
Guided Electronic Therapy in the Treatment of Depression
The delivery of web-based therapy as a treatment for depression can be performed in one of two ways: through the use of supports to assist patients through the web-based therapy (guided model) or through the self-help use of computerized treatment (unguided model). In the guided model, patients are provided support as they progress through web-based therapy. In some cases, highly trained clinicians have been used to fill this role ; however, their advanced training is costly and does not appear to provide any added benefit. For instance, despite the use of a clinician in the delivery of web-based therapy for the treatment of depression and anxiety among young adults, Dear et al [ ] did not find any significant differences in symptomatology when compared with the unguided condition. One solution that has been implemented to mitigate this issue is the use of coaches to support patients progressing through web-based therapy, answering both technical questions about how the program works as well as providing support and encouragement. To date, coaches have included both students [ ] and licensed professionals from a variety of backgrounds, including psychology [ ], counseling [ ], and social work [ , ]. The components of effective coaching are still uncertain [ ], with most of the literature emphasizing the technical aspects of an internet intervention. It is not clear whether the professional background of the coach, the frequency of contact, or the content of the coaching sessions affects treatment outcomes.
Several systematic reviews have demonstrated that internet-based interventions for depression have effect sizes that are comparable with face-to-face interventions, whereas unguided interventions have smaller effect sizes [, , ]. However, the evidence is inconsistent, with head-to-head comparisons of guided versus unguided interventions showing mixed results. The largest study of guided web-based therapy compared with usual care has been the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) study [ ], which was a UK-based RCT of 691 patients with depression in primary care. Participants were randomized to one of three treatment groups: (1) usual care; (2) Beating the Blues, a guided commercial web-based program; or (3) MoodGYM, a guided free web-based program. The study found no difference in depression outcomes between the three groups. Similarly, Kenter et al [ ] compared student-assisted internet-based problem-solving therapy with an information-only waitlist control and found minimal differences in depression severity (Cohen d=0.07), as measured by the Centre for Epidemiology Studies Depression scale.
Adherence to Web-Based Therapy Treatment Protocols
Web-based interventions for depression struggle with engagement and adherence to treatment protocols. For example, in the Beating the Blues and MoodGYM arms of the REEACT trial, participants completed only a median of 1 to 2 sessions and received only 6 min of technical support time, 5 emails, and almost no text messages from the telephone support workers. Further, approximately 1 in 5 participants randomized to either of the web-based therapy conditions did not access the programs at all. Similar problems were encountered in the study conducted by Kenter et al , with only 36% of participants receiving an adequate dosage of the intervention, defined by the authors as 4 of 5 lessons of web-based therapy. From a public health perspective, low adherence to web-based therapies dilutes their effectiveness. However, it is unclear if, for individual patients, adhering to a complete course of web-based therapy is better than control treatments.
We have previously reported on an RCT of The Journal comparing guided web-based therapy with an information-only waitlist control in patients with depression referred to secondary mental health services in New Zealand . Participants were recruited face-to-face during their triage visit at community mental health centers, and the study found no difference in depression or service use after 12 weeks. However, all participants in this study were also receiving mental health care from clinicians, which may have outweighed any effect of the web-based therapy. In this study, we report on an RCT of The Journal [ ], facilitated by a coach, compared with an information-only waitlist control group for the treatment of depression in patients referred to secondary mental health services in Canada. In this setting, access is a significant issue, with patients often waiting for over a year to access mental health services. During this time, patients on the waiting list do not receive any other mental health care other than routine follow-up from their family physician; therefore, the provision of web-based therapy could potentially be used as an alternative to referral to specialized mental health services. This study gave us an opportunity to refine the study intervention. In contrast to the New Zealand study, we opted to use an information-only control group as it is a low-cost, low-risk alternative that could easily be implemented in clinical practice. Similarly, we were able to refine the coaching aspect of the intervention. We hypothesized that, after 12 weeks of treatment, participants receiving coach-guided web-based therapy would experience a greater reduction in depressive symptoms and health service use.
The design of the trial was an RCT with two parallel groups. This trial has been reported according to the Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth (CONSORT-EHEALTH;).
Potential participants were patients referred to the Royal Ottawa Mental Health Centre (ROMHC, Ottawa, Canada) with symptoms of depression or dysthymia who were on a waiting list for treatment in the following psychiatric programs: Mood and Anxiety, Geriatric Psychiatry and Youth Psychiatry. The ROMHC has 284 inpatient beds and acts as a specialized mental health facility for residents of communities across Eastern Ontario. These programs are aimed at treating people with complex and serious mental illnesses that are often resistant to treatment. Patients are referred to treatment in these programs directly by their family physicians. The first appointment in the programs is with a psychiatrist who then decides on a treatment plan with the patient. At the time of this clinical trial, the waitlist for the Mood and Anxiety Program was between 9 months and 1 year.
Eligibility criteria for participation in the trial are outlined in. Potentially eligible patients were contacted using the following methods: patients who had completed a consent to be contacted for research as part of their referral documentation were contacted by telephone. Those who did not complete the consent to be contacted for the research section of the referral were contacted by mail, as per institutional policies. Interested patients were preliminarily screened by a research assistant for eligibility to participate in the study. Eligible patients were then asked to attend a face-to-face consenting appointment with a research assistant. A minority of patients who could not travel to the clinic were asked to consent via mail (n=3).
Participant eligibility criteria.
- 16 years of age or older.
- Referred and triaged to the Mood and Anxiety, Youth Psychiatry, or Geriatric Psychiatry Program at the Royal Ottawa Mental Health Centre for any depressive symptoms.
- Willing to attend electronic therapy sessions for up to 12 weeks.
- Able and willing to provide informed consent.
- Willing to be randomized.
- Willing to comply with all study procedures.
- Is unable to read or write in English.
- Does not have an Ontario Health Insurance Plan number.
- Has cognitive impairments that render them unable to use a computer.
- There is another participant enrolled in the study who lives at their address.
The Journal  is an evidence-based free web-based therapy program developed in New Zealand for the self-management of depression that utilizes the cognitive behavioral techniques of behavioral activation and problem solving ( ). The problem-solving approach was derived from a large RCT of face-to-face problem-solving used in people who presented to emergency departments with intentional self-harm [ ].
As described in, there are a total of 9 modules in The Journal, 6 of which must be done for patients to complete the program. Participants progressed through the web-based therapy as follows: (1) positivity module (1 lesson); (2) lifestyle modules (4 lessons, 1 of which must be completed); and (3) problem-solving module (4 lessons). At the start of each lesson, users are asked to watch a video featuring Sir John Kirwan, a former All-Blacks rugby player, who has been very public about his own struggles with depression in an effort to reduce stigmatization.
|Lessons||Description||Tasks to be completed by participants|
|Positivity module||Learn the importance of staying positive and planning regular activities that they enjoy|
|Eating right||Explore the link between diet and mood.|
|Getting active||Review benefits of being active on mood.|
|Learning to relax||Highlights the importance of stress management.|
|Sleeping better||Discuss the importance of good sleep habits to mood.|
|Identify problems||Learn how depression impacts problem-solving abilities.|
|Find solutions||Explore how to use both logical and creative parts of the brain to brainstorm problem solutions.|
|Create a plan||Review how to create SMARTb plan.|
|Review your plan||Highlights the importance of assessing progress and updating the SMART plan.|
aParticipants are only required to complete 1 of the 4 lifestyle lessons.
bSMART: specific, measurable, achievable, relevant, time-bound.
Both groups received usual care while on the waitlist, which included management by a family physician and use of community resources, such as access to distress center lines and counseling services. Once participants are called off the waitlist, they receive an initial appointment with a psychiatrist, at which point adjustments are made to the patient’s care plan. They are then referred back to their family physician, or they receive further psychiatric treatment (eg, regular care from a psychiatrist, social worker, nurse, occupational therapist, and so on).
Following the consent appointment, participants were randomized to one of two treatment groups. In addition to usual care, participants assigned to the control group received an information leaflet with web-based resources, including The Journal, and told that they could decide for themselves the best way to use this information while on the waitlist. Participants had been previously informed by their clinical team that the estimated wait time to be seen by a psychiatrist was between 9 and 12 months.
For those participants assigned to the intervention group, the intervention consisted of the following:
- An information leaflet of web-based depression resources.
- An invitation to use The Journal.
- 12-weekly telephone coaching sessions with a coach (SL), who had a guideline script for each coaching session, reinforced the topic of each lesson in The Journal, helped identify and support participants in goal setting and the techniques of problem-solving. Each session lasted between 30 and 60 min. The coach had a background in social work and received weekly supervision from the principal investigator (SH).
- Text message or email contact between appointments, as per the participant’s preference.
The primary outcome measure was the Patient Health Questionnaire (PHQ-9) , a 9-item questionnaire that assesses the severity of depression symptoms experienced within the preceding 2 weeks. Participants are asked to rate each symptom of depression on a Likert scale from 0 (not at all) to 3 (nearly every day), with total scores ranging from 0 (minimal depression) to 27 (severe depression). The PHQ-9 has strong methodological properties, with an internal consistency of 0.89 and strong test-retest reliability [ ]. Increasing scores on the PHQ-9 have also been found to be correlated with deteriorating scores on all 6 subscales of the Medical Outcomes Survey Short Form-20 [ ]. The PHQ-9 was selected not only for its strong psychometric properties but also for its commonality. The PHQ-9 is often used as a screening tool for major depressive disorder in primary care practice [ ].
Suicidal thoughts were assessed by question 9 of the PHQ-9, in which respondents were asked Over the last 2 weeks how often have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way? . This variable was dichotomized as follows: participants who responded Not at All (0) were categorized as no (0), and participants who reported any degree of suicidality (1, 2, or 3) were categorized as yes (1).
Health-related quality of life was assessed using the EuroQol-5 dimension (3 levels) questionnaire (EQ-5D-3L). This is a 5-item questionnaire that assesses health-related quality of life, including mobility, self-care, ability to participate in one’s usual activities, pain or discomfort, and anxiety or depression. The EQ-5D-3L asks participants to assess their health-related quality of life on a 3-point scale from no dysfunction to extreme dysfunction, with the following response categories:
- Level 1: indicating no problem.
- Level 2: indicating some problems.
- Level 3: indicating extreme problems.
The EQ-5D-3L is then able to define a unique health state based on the responses to each of the 5 dimensions of health described above. Respondents fall into 1 of 243 different health states, depending on their responses to the questionnaire . For instance, an overall score of 11111 indicates no problems in any of the 5 health dimensions, whereas a score of 12312 indicates that a respondent has no problems with mobility, some problems with washing or dressing, extreme problems with doing usual activities, no pain or discomfort, and some anxiety or depression. The measure also includes a visual analogue scale (VAS), which asks participants to evaluate their overall health on a scale from 0 to 100. The EQ-5D-3L has strong psychometric properties and has been found to be moderately to highly correlated with other measures of impairment and disability [ , ].
Service use was measured using data extracted from participants’ electronic medical records (EMR), including, time to first consultation appointment at the ROMHC and the total number of outpatient mental health follow-up appointments after the first consultation completed at the ROMHC. These measures were administered as shown in.
Baseline assessments were administered in-person following the consent appointment, and all other time point assessments were conducted by telephone either by the coach (intervention group) or a research assistant (control group). Patients who missed appointments or were lost to follow-up were also sent questionnaires by mail. Service use at the ROMHC was obtained from the EMR of each participant. Five participants in the control group were excluded ad hoc to prevent confounding as they accessed The Journal during the treatment period.
|Variable||Outcome measure||Time point|
|Depressive symptoms||PHQ-9a||Baseline, week 2, week 6, week 12|
|Suicidal thoughts||PHQ-9 Q9b||Baseline, week 2, week 6, week 12|
|Health-related quality of life||EQ-5D-3Lc, EQ-5D-VASd||Baseline, week 6, week 12|
|Health service use||ROMHC EMRe||One year following the initial consultation appointment at the ROMHC|
aPHQ-9: Patient Health Questionnaire.
bPHQ-9 Q9: Patient Health Questionnaire question 9.
cEQ-5D-3L: EuroQol 5 dimensions (3 levels) questionnaire.
dEQ-5D-VAS: EuroQol 5 dimensions visual analogue scale.
eROMHC EMR: Royal Ottawa Mental Health Center electronic medical record.
Based on the previous studies that used the PHQ-9 as their primary outcome measure, we expected the mean pretreatment score to be 17.0 (SD 4.0). To detect a difference in PHQ-9 scores between the two groups of at least 3 points, an established minimal clinically important difference , we would need 44 participants in each group with a 2-sided alpha of .05, and a power of 80.0% (effect size of 0.6). Allowing for a 25.0% dropout rate, we aimed to recruit a total of 110 participants.
Randomization was completed by the Ottawa Methods Centre at the Ottawa Hospital Research Institute, with allocations kept in sequential sealed envelopes at the study base. Participants were randomized in a 1:1 allocation, and there were no restrictions. After providing consent, participants were randomized by a research assistant according to the allocation in the sealed envelopes.
Owing to the nature of the intervention, neither participants nor study staff were blinded to the treatment allocation. Outcome assessments were collected by delegated study staff who were not blinded to the treatment allocation.
Group differences in demographic and pretreatment measures were analyzed using independent samples t tests. Changes in participants’ scores from pretreatment to follow-up at 12 weeks were assessed using repeated measures analysis of variance with mixed linear modeling to account for missing variables. This model included the following variables: treatment group (control or intervention), PHQ-9 scores at 4 different time points (baseline, week 2, week 6, and week 12), gender (male or female), and age. PHQ-9 scores were entered as within-subject variables, treatment group as between-subject factors, and age and gender as covariates. Statistical analyses were conducted using the general linear modeling repeated measures procedure in IBM SPSS Statistics 25 for Windows. To assess significant differences in outcome measures, the last observations for the PHQ-9, EQ-5D-3L and EQ-5D-VAS were carried forward for the 6- and 12-week time points.
Differences in suicidal thoughts between the two groups were assessed using an independent samples t test. Differences in proportions of service use were assessed using chi-square tests. In addition, we also assessed whether participants experienced a clinically important reduction in depression symptoms defined as a PHQ-9 score of 9 or less or a 50.0% reduction in scores . This was described using percentages and frequencies, and chi-square was used to assess the significance of the differences in proportions. Assessments of normality were completed using the Shapiro-Wilk test for continuous data, and the Mann-Whitney U test for nonparametric data.
As per the recommendations outlined in the Medical Research Council’s guidelines for the assessment of complex interventions , we conducted a process evaluation to assess the context, implementation, and mechanisms of impact of the study intervention. The process evaluation outcome measures are outlined in . We also conducted semistructured interviews with participants within 6 months of study completion to assess their experience of the study intervention, including access to and functionality of The Journal, the therapeutic content and value of The Journal, and the experience of working with a coach. Interviewees were identified using a purposive sampling approach and were stratified to include participants in both arms of the trial and varying levels of engagement (eg, not engaged, moderately engaged, and highly engaged). Interviews were conducted until data saturation was reached (n=15).
|Evaluation critierion and outcome measure||Description|
|Facilitators of and barriers to study completion|
|Mechanisms of impact|
|Role of the coach|
aAll usage data from The Journal were assessed via participant self-report.
Individual interviews were conducted by two female University of Ottawa Department of Psychiatry residents (JK and ZS) who were independent of the research team that conducted the RCT. Following transcription, two independent coders (JK and KG) analyzed the material using a thematic, grounded theory approach. The coding took place in two stages, with coders meeting during the first stage to discuss emergent codes, reconcile definitions, and compare coding rationales. The two coders had not been involved in delivering the treatment or any of the previous study contacts.
The study received approval from the Royal Research Ethics Board (Protocol 2014001). All participants provided informed consent before participation in both the RCT and qualitative interviews.
Recruitment for this study took place over 11 months, from May 2015 to April 2016, and a total of 1316 patients were preliminarily screened for eligibility by examining their referral documentation (). Of these, 45.9% (605/1316) could not be reached to complete a full screening for eligibility; 40.4% (532/1316) could be contacted were not eligible to participate, and 13.6% (179/1316) were eligible and could be contacted.
Of the 45.9% (605/1316) of patients who could not be contacted to complete a full eligibility screen, 25.4% (154/605) provided consent to be approached about research studies but did not reply to telephone or mail invitations; 63.1% (382/605) were invited to discuss the study by mail with no response, and 11.3% (69/606) had incomplete mailing information listed on their referral documents.
Of the 40.4% (532/1316) of patients who could be contacted but were ineligible to participate, 84.9% (452/532) were referred for a reason other than depressive symptoms; 5.6% (30/532) had cognitive impairments rendering them unable to use a computer; 4.5% (24/532) were no longer on the waitlist at the time of screening; 2.6% (14/532) were under 16 years of age at the time of screening; 1.12% (6/532) were unable to read and write in English; and, 1.12% (6/532) did not have a valid Ontario Health Insurance Plan number.
Of the 13.6% (179/1316) of patients who were eligible to participate, 46.9% (84/179) declined to participate, and 53.1% (95/179) consented to participate in the study.lists the reasons for patients declining to participate.
The majority of participants enrolled in the study were recruited from the Mood and Anxiety Outpatient Clinic (n=92), and 3 were recruited from the Geriatric Psychiatry Program at the ROMHC. No participants were enrolled from the Youth Psychiatry Program.describes the demographic characteristics of the participants. There were significantly more women in the intervention group and more men in the control group (Χ21=6.6; P=.01).
Participants in the intervention group completed a mean number of 5.0 (SD 2.3) lessons in The Journal and 8.8 (SD 3.1) sessions with the coach. In the control group, 10% (5/48) of participants reported accessing The Journal. In the control group, 17% (8/48) of participants scored 9 or below on the PHQ-9 at baseline compared with 23% (11/47) of participants in the intervention group. Owing to gender imbalances between the groups, we also conducted a post hoc gender analysis of changes in PHQ-9 scores and health service use.
|Reason for Nonparticipation||Value, n (%)|
|Did not attend baseline intake appointment||27 (32)|
|Did not have computer/internet at home—not interested in going to public library or community center||16 (19)|
|No reason provided||13 (15)|
|Interested in participating in a different study also recruiting from the mood and anxiety program||6 (7)|
|Too overwhelming||4 (5)|
|Would prefer to wait for appointment with psychiatrist||3 (4)|
|Not interested in participating in research at the Royal Ottawa Hospital||3 (4)|
|Did not feel that the study would benefit them||2 (4)|
|Not interested in weekly contact||2 (2)|
|No time||2 (2)|
|Family circumstances||1 (1)|
|Participating in another research study||1 (1)|
|Interested in medication change or recommendations||1 (1)|
|Migraines due to computer use||1 (1)|
|Moving out of province||1 (1)|
|Interested only in face-to-face therapy||1 (1)|
|Demographic characteristic||Total (n=95)||Control group (n=48)||Intervention group (n=47)|
|Gender, n (%)a|
|Male||28 (30)||20 (42)||8 (17)|
|Female||66 (70)||28 (58)||38 (83)b|
|Age (years), mean (SD)||44.2 (12.9)||44.8 (13.7)||43.5 (12.1)|
|Ethnicity, n (%)|
|First Nations||1 (1)||0 (0)||1 (2)|
|Inuk||0 (0)||0 (0)||0 (0)|
|Métis||3 (3)||3 (6)||0 (0)|
|Asian||3 (3)||2 (4)||1 (2)|
|African, Caribbean, or Black||0 (0)||0 (0)||0 (0)|
|White||82 (87)||42 (88)||40 (87)|
|Other||5 (5)||1 (2)||4 (9)|
|Marital status, n (%)|
|Single||33 (35)||11 (23)||22 (47)|
|Common law||6 (6)||4 (8)||2 (4)|
|Married||35 (37)||18 (38)||17 (36)|
|Separated||4 (4)||3 (6)||1 (2)|
|Divorced||16 (17)||11 (23)||5 (11)|
|Widowed||1 (1)||1 (2)||0 (0)|
|Education level, n (%)|
|High School||10 (11)||7 (15)||3 (6)|
|College||45 (47)||23 (48)||22 (47)|
|University—undergraduate||26 (27)||10 (21)||16 (34)|
|University—master’s||10 (11)||5 (10)||5 (11)|
|University—doctorate||4 (4)||3 (6)||1 (2)|
|Employment status, n (%)|
|Full-time||23 (24)||11 (23)||12 (26)|
|Part-time||8 (8)||2 (4)||6 (13)|
|Short-term disability||5 (5)||2 (4)||3 (6)|
|Long-term disability||31 (33)||18 (38)||13 (28)|
|Self-employed||7 (7)||3 (6)||4 (8)|
|Retired||10 (11)||5 (10)||5 (11)|
|Unemployed||11 (12)||7 (15)||4 (8)|
an=1 transgender participant removed from the analysis.
The mean PHQ-9 score was lower in the intervention group than in the control group at all study time points (). The proportions of missing values were comparable between the 2 groups at all assessment points, with the exception of week 12 (2/46 , 4% in the control group compared with 0/47, 0% in the intervention group). The Shapiro-Wilk test of normality revealed that all data were normally distributed with the exception of the control group at the week 12 study time point (W46=0.95; P=.04).
At 12 weeks, the mean PHQ-9 score in the intervention group was 11.3 (SD 6.4) and was 12.4 (SD 6.4) in the control group (t86=0.76; P=.45, 95% CI −1.9 to 3.8). To account for the non-normality in scores at week 12, mean differences (MDs) at this time were also assessed using a Mann-Whitney U test of significance, which confirmed that the difference between the two groups was not statistically significant (Mann-Whitney U statistic=862.0; Z=−0.85; P=.40). The mean change in PHQ-9 scores from baseline to week 12 was −3.6 (SD 6.6) in the intervention group and −3.1 (SD 6.2) in the control group, which was not statistically significant (t91=−0.37; P=.72, 95% CI −3.1 to 2.2).
Excluding the 10% (5/48) of participants in the control group who also accessed The Journal did not result in any statistically significant differences in mean PHQ-9 scores compared with the intervention group. The mean score at week 12 for the control group excluding those who accessed The Journal (n=41) was 12.4 (SD 6.2) compared with 10.7 in the intervention group (t86=0.76; P=.45, 95% CI −0.9 to 4.3).
|Study time point||Mean (SD)||Median||Missing values, n (%)||Mean difference (95% CI)||Independent samples t test (df)||P value|
|Baseline||0.5 (−1.8 to 2.9)||0.45 (93)||.66|
|Control (n=48)||15.4 (5.4)||16.5||0 (0)|
|Intervention (n=47)||14.9 (6.0)||16.0||0 (0)|
|Week 2||1.1 (−1.3 to 3.5)||0.92 (89)||.36|
|Control (n=46)||12.6 (5.5)||11.5||2 (4)|
|Intervention (n=45)||11.5 (5.9)||12.0||2 (4)|
|Week 6||1.7 (−0.9 to 4.3)||1.29 (89)||.19|
|Control (n=46)||12.4 (6.2)||12.0||2 (4)|
|Intervention (n=45)||10.7 (6.1)||10.0||2 (4)|
|Week 12||1.0 (−1.9 to 3.8)||0.76 (89)||.45|
|Control (n=46)||12.4 (6.4)||11.5||2 (4)|
|Intervention (n=47)||11.3 (6.4)||10||0 (0)|
The repeated measures modeling found that scores on the PHQ-9 differed significantly by study time point, irrespective of group allocation (F2.6,222.8=11.59, P<.001). However, this relationship was not sustained once treatment group was taken into account (F1,87=1.46, P=.23). Similarly, modeling found that scores on the PHQ-9 were not significantly associated with gender (F1,87=0.33, P=.57) or the interaction between treatment group and PHQ-9 scores (F1,87=0.70; P=.53). However, results were significantly associated with age of participants (F1,87=5.97, P=.02, ŋ2=0.06).
Health-Related Quality of Life
After 12 weeks of treatment, participants in the intervention group reported higher health-related quality of life, with mean index scores on the EQ-5D-3L of 0.7 (SD 0.7) for the intervention group and 0.6 (SD 0.5) for the control group (t86=−2.30; P=.02, CI 95% −0.2 to −0.1;). Similarly, at 12 weeks, the mean EQ-5D-VAS score was significantly greater in the intervention group (mean 66.8, SD 18.0) than in the control group (mean 55.9, SD 19.2; t84=−2.73; P=.01).
Changes in Suicidality
At 12 weeks, of the 47 participants in the intervention group, 66% (31/47) reported no suicidality, 28% (13/47) reported suicidality several days in the preceding 12 weeks, 2% (1/47) reported suicidality nearly half the days, and 4% (2/47) reported suicidality nearly every day. In comparison, of the 46 participants in the control group, 69% (33/46) reported no suicidality, 17% (8/46) reported suicidality several days, 6% (3/46) reported suicidality more than half the days, and 4% (2/46) reported suicidality nearly every day (). These differences in suicidality were not statistically significant (Χ23=2.2; P=.52).
|Study time point||Control (n=48), n (%)||Intervention (n=47), n (%)||Chi-square (df)||P value|
|Not at all||24 (50)||23 (49)|
|Several days||16 (33)||18 (38)|
|More than half the days||5 (11)||4 (9)|
|Nearly everyday||3 (6)||2 (4)|
|Total||48 (100.0)||47 (100.0)|
|Week 2||1.37 (3)||.71|
|Not at all||30 (63)||34 (72)|
|Several days||12 (25)||8 (17)|
|More than half the days||2 (4)||2 (4)|
|Nearly everyday||2 (4)||1 (2)|
|Total||46 (96)||45 (95)|
|Week 6||4.70 (3)||.20|
|Not at all||27 (56)||32 (68)|
|Several days||11 (23)||11 (23)|
|More than half the days||5 (10)||2 (4)|
|Nearly everyday||3 (6)||0 (0.0)|
|Total||46 (95)||45 (95)|
|Week 12||2.24 (3)||.52|
|Not at all||33 (69)||31 (66)|
|Several days||8 (17)||13 (28)|
|More than half the days||3 (6)||1 (2)|
|Nearly everyday||2 (4)||2 (4)|
|Total||46 (96)||47 (100.0)|
Hospital Service Use
Participants in both groups received similar levels of face-to-face follow-up care, with 25.0% of participants in the intervention group receiving any follow-up in the next 12 months at the hospital after their initial outpatient appointment with a psychiatrist compared with 21.3% of the control group (Χ21=0.7; P=.80;). Participants in the intervention group attended a mean number of 4.8 (SD 8.5) face-to-face follow-up appointments compared with 3.6 (SD 5.6) in the control group (Mann-Whitney U statistic=255.50; Z=−0.11; P=.91).
|Health service use indicator||Control||Intervention||Tests of association||P value|
|Number of days from referral to first appointmenta, mean (SD)||213.6 (54.6)||219.3 (57.0)||t69=−0.43 P||.67|
|Number of people who received outpatient follow-up by a nonpsychiatrist after initial assessment by a psychiatrist, n (%)||13/44 (30)||12/46 (26)||Χ21=0.1||.71|
|Number of people who received outpatient follow-up by a psychiatrist after their initial consultation, n (%)||20/44 (42)||18/46 (38)||Χ21=0.3||.54|
|Number of outpatient follow-up appointments with a psychiatrist in the year after the initial consultationb, mean (SD)||2.3 (3.1)||2.5 (4.6)||t88=−0.30||.76|
|Number of outpatient follow-up appointments with all disciplines in the year after initial consultationc, mean (SD)||3.6 (5.6)||4.8 (8.5)||M-Wd U=255.5, Z=−0.11||.91|
aControl group n=34; Intervention group n=37.
bControl group n=34; Intervention group n=37.
cControl group n=34; Intervention group n=37.
As reported above, only 13.4% (95/711) of potentially eligible patients (eg, those who could be contacted and were on the waitlist for mental health treatment) agreed to participate in the trial. Of particular note is that 8.9% (16/179) eligible participants declined to participate because they did not have computer or internet access at home. This is comparable with 14% of the Canadian population . Similarly, among participants who were randomized to one of the trial arms, 42% (40/95) were university-educated compared with 33.3% of residents of Ottawa [ ].
Analysis of trial records demonstrated that the study intervention was implemented as intended in the study protocol. Participants in the intervention group completed a mean of 5.0 (SD 2.3) lessons in The Journal. Similarly, throughout the course of the study, participants completed a mean of 8.8 (SD 3.1) telephone coaching sessions, ranging from a mean of 9.6 to 61.6 min in duration, and a mean of 21.6 (SD 10.8) contacts with the coach (mean 13.1 (SD 4.0) telephone calls, 2.0 (SD 2.5) emails, and mean 25.7 (SD 15.4) text messages. All fidelity measures were significantly associated with PHQ-9 scores at 12 weeks, with the exception of the types of contact with the coach (). Total lessons completed in The Journal, total number of coaching sessions completed, and average length of coaching calls were all inversely related to PHQ-9 scores at 12 weeks, with those having completed more lessons in The Journal, a higher number of sessions with the coach and longer coaching calls reporting lower levels of depression after 12 weeks of treatment ( ).
|Fidelity Measures||Mean (SD)||Median||Pearson’s correlation with PHQ-9a scores at week 12, (n=47)|
|Total lessons completed in The Journal||5.0 (2.3)||6.0||−0.436||.002|
|Total sessions with the coach||8.8 (3.1)||10.0||−0.435||.002|
|Average length of coaching calls (min)||30.8 (12.9)||27.7||−0.360||.01|
|Total contacts with coach by type|
aPHQ-9: Patient Health Questionnaire.
For those participants in the intervention group who received a full dose of the study intervention, completing 6 or more lessons from The Journal (n=29), the mean PHQ-9 scores at 12 weeks was 9.5 (SD 5.7) compared with 14.2 (SD 6.6) among those who completed 5 lessons or less in The Journal and 12.4 (SD 6.4) in the control group (F2,85=3.5; P=.04). Post hoc comparisons using the Tukey Honestly Significant Difference test indicated that the MD among those who completed at least 6 lessons in The Journal was significantly different from those who completed 5 or fewer lessons in The Journal (MD=−4.7, P=.04). However, the MDs between the control group and both experimental groups were not statistically significant (MD≥6 Lessons=−2.8; P=.15; MD≤5 Lessons=1.86; P=.55). Similarly, those who completed at least six or more lessons in The Journal were more likely to demonstrate a clinically significant reduction in symptoms, with 65.5% reporting a significant reduction in symptoms compared with 27.8% of those who completed less than 5 or fewer lessons and 37.0% of the control group (Χ22=8.2; P=.02; Φ=0.30).
Similarly, by week 12 in the intervention group, 51% (24/47) of participants had a clinically important reduction in depressive symptoms (PHQ-9 score of 9 or less or a 50% or more improvement in scores) compared with 37% (17/46) of participants in the control group. However, this difference was not statistically significant (Χ21=1.8; P=.17). Among participants in the intervention group who completed the full dose of the intervention by finishing 6 or more Journal sessions, 66% (19/29) had clinically important reductions in depression at 12 weeks compared with 37% (17/46) in the control group (Χ21=6.3; P=.01; Φ=0.37).
Context and Mechanisms of Impact
A total of 15 qualitative interviews were conducted (10 intervention, 5 control) with 4 male (2 intervention group, 2 control group), 10 female (7 intervention group, 3 control group), and 1 transgender person (intervention group). Interviews took place at the ROMHC at a time that was mutually convenient for both participants and interviewers. Themes that arose from the interviewers were categorized in terms of whether there were positive facilitators of or negative barriers to engagement with the study intervention.
Facilitators to Engagement With The Journal
Participants identified Sir John Kirwan’s relatability as a positive facilitator through his sharing of his personal experience with depression. For instance, a participant commented on the videos that begin each lesson in The Journal, explaining the following:
…I just like the way that they interacted with each other and how casual it was, so it didn’t feel like I was doing homework, and it didn’t feel like I was doing medical stuff…It was just, watching two people sitting on a bench in a park and they were talking, and I really liked how the narrator came right out and said, you know, when I was depressed, you know, I couldn’t get out of bed, or I, I didn’t want to take a shower, it was, like, wow somebody else feels the way I do. So, it helped me to feel like I wasn’t all, all alone. And he was, he seemed to be very honest about his experience.
Similarly, participants expressed an appreciation for the structure and layout of The Journal, specifically commenting on the usefulness of the Problem-Solving modules:
Interviewer: …what did you like about the problem-solving lesson?
Participant: … it was in a new, new approaches. It gave me a new technique or I found a new technique in there in terms of outlining problems, and thinking about them and, that I hadn’t.
Barriers to Engagement With The Journal
Participants also reported frustrations with some aspects of The Journal, specifically relating to the technical issues they encountered. At the time of the trial, The Journal was not available on mobile devices, such as smartphones or tablets, which may have limited participants’ ability to access it. Similarly, technical issues presented themselves within the web-based program as well. For instance, in each lesson, participants must set deadlines for tasks that they are required to complete and are unable to move forward in The Journal until these deadlines pass. Participants also identified issues with motivation and questioned whether The Journal would be most appropriate for patients who are only mildly depressed. For instance, one participant explained:
Participant: Yeah, lifestyle one…was little bit of a challenge.
Interviewer: Yeah? In what way?
Participant: … just trying to find motivation…it’s just hard to change some things about lifestyle, but it came with time”
Similarly, a participant in the control group commented on how some depressive symptoms, such as difficulty with concentration, may act as barriers to engagement with treatment:
Interviewer: … Were there any barriers or challenges to using the brochure?
Participant: …not really because it’s short and sweet…my only barrier was time, you know, time and, and energy. Because at the time, and it was almost an opportune time that I was doing this because I was low at the time… so when you are depressed the last thing you want to do it read, so you can’t even. And that’s why I was saying that if this was little bullet chunks, you know, you don’t have to read it so much, cause I can’t read a book when I’m down. Like I start to read a paragraph and I’m like, no I have to close it.
Coach as a Mechanism of Change
Numerous participants in the intervention group also identified the coach as key to their success with the program, highlighting the importance of accountability:
Interviewer: …what would you say made the biggest difference to your participation in the study? ...was it the journal, was it the coach?
Participant: It was the coach, oh my god. I would not have stayed with that journal unless [SL] was calling me… cause then I wouldn’t really have anybody to be like, hey did you do that thing?
Similarly, another participant questioned whether they would have been as successful in completing the program had they not had someone following-up on their goals:
…if it was by myself, I might have just kept postponing it or not doing it or, so I found it quite helpful. She's a pretty great coach.
Similarly, another participant highlighted the importance of continuity between the lessons in The Journal and the content of the coaching sessions:
I like the conversations the following week with the homework that we did. I liked that it was continuous.
Accountability also emerged as important for participants in the control group. For instance, 1 participant who accessed The Journal expressed a lack of follow-up by a third party as a barrier to completing the program, suggesting that:
…you guys can follow-up and see if they’ re.. If people are using it or not…just to say, somebody is following me, I should probably do a little bit more of this...
This was echoed by another participant in the control group who struggled with the limited follow-up they received from the research team:
taking not even five minutes to say “how are you doing right now?”… So in a way it gauges their, their level of health, of mental health at the time, and if they do appear to need resources then [the Research Assistant] could say “look I’m not a doctor, I’m not a psychiatrist, but do you remember the flyer that I gave you, on it there was this thing that might be helpful for you right now, you might want to give it a thought”… so, that’s just like extra, like value added for resourcefulness.
Among patients on a waiting list referred to secondary care for depressive symptoms, a trial of usual care plus coach-facilitated web-based therapy compared with usual care plus information-only waitlist control group found no statistically significant differences in mean depression scores after 12 weeks. However, participants who completed 6 or more lessons in The Journal reported significantly greater reductions in symptoms compared with controls. Participants in the intervention group also reported better quality of life after 12 weeks of treatment. There was no difference in service use between the two groups. During the qualitative interviews conducted as part of the process evaluation, participants identified the role of the coach as a major factor in their completion of the web-based therapy program.
Strengths and Limitations
The major strength of this study is that it is the first study of coach-assisted web-based therapy in a secondary care setting in Canada. This study demonstrated that it is possible to incorporate the use of a coach within this clinical setting. However, due to limitations imposed by institutional policies, study staff were restricted in their ability to recruit participants, with only 12% (95/784) of potentially eligible patients agreeing to participate in the trial. It is reasonable to expect greater reach if this intervention was rolled out by a clinical service that could approach people directly at the time of referral.
There was also a possibility of contamination between the arms of the trial. Given that The Journal is a publicly accessible website, any participant was able to access it, and there was little that could be done by the study team to prevent this. However, our experience is that patients in secondary care do not widely use The Journal, with only 10% (5/48) participants in the control group reporting use of The Journal during the treatment period. Furthermore, contamination is likely to bias the study to showing no difference (as the control group could use The Journal unguided), so any differences that are found are likely to be more believable. Finally, the ability to generalize study findings to the larger Canadian population may be limited as a result of our highly educated population.
Some participants who were interviewed as part of the process evaluation expressed issues with recall given that the interviews took place 6 months after study completion. It is possible that interviews conducted closer to study completion would have yielded richer qualitative data.
Comparison With Prior Work
A key contribution of this study is that participant engagement with and adherence to the intervention was much higher than that reported in previous work, with 62% (29/47) of participants in the intervention group receiving a full dose of treatment, completing an average of 5 out of 6 lessons in The Journal and 8.8 telephone coaching sessions with the coach. Comparatively, in the REEACT trial, participants only completed a median of 2 sessions of web-based therapy. Similarly, in the study by Kenter et al , only 36% of participants received the required 4 to 5 sessions of internet-based psychotherapy. This was found in spite of difficulties with ease of access, as The Journal was not available on mobile or tablet devices at the time of the study. This trial also highlights the importance of the uptake of the intervention to clinical outcomes, with participants who completed at least 6 lessons in The Journal experiencing a significantly greater reduction in depression symptoms than those who did not.
The previous trial of a guided version of The Journal in New Zealand secondary care showed no differences in clinical outcomes or service use. However, the patients in the New Zealand study had higher baseline depression scores, a mean PHQ-9 of 17.0, and were less engaged with the study intervention, completing only a mean of 2.5 lessons in The Journal. The magnitude of change in PHQ-9 scores was also different, with an intervention group change in PHQ-9 scores of 3.5 in Canada compared with 9.4 in New Zealand. This presumably reflects differences in usual care, with participants in New Zealand receiving mental health care, whereas Canadian participants received care from their primary care physicians while on a waiting list.
Web-based therapies are often promoted as a way to address long waiting lists for mental health care services. However, in this study, even though participants in the intervention group with more exposure to the study intervention were more likely to experience a significant reduction in symptoms, this had little impact on subsequent service use. Web-based therapies are part of a complex socio-technological system and, as such, cannot exist in a vacuum. To achieve improvement in patient outcomes, they must be integrated into a larger system of care. This could be achieved with a more explicit stepped care system supported by web-based therapies, with a more flexible response from providers based on patient need.
Unanswered Questions and Future Research
This study provides limited support for the potential use of web-based therapies within a stepped-care approach to the treatment of depression. However, an RCT is needed to determine the effectiveness of such an approach. The impact of providing digital services to those in greatest need, who are also the least likely to have access to high-speed connections, also needs to be taken into account. Future research on internet-based psychotherapy for depression needs to include the system of care in which it is used. This can be done through implementation science tools, which not only evaluate the effectiveness of web-based therapies but also the factors central to their uptake, such as reach, adoption, and sustainability.
The results of this study demonstrated that the use of guided web-based therapy for the treatment of depression is not more effective than information-only waitlist control. However, it showed that coach-guided web-based therapy has the potential to increase adherence and engagement with depression treatment protocols. Greater adherence resulted in greater effectiveness. More research is needed on the human component of coaching in conjunction with web-based therapy to examine what factors lead to greater adherence. Researchers also need to consider when and how web-based therapies should be integrated into existing clinical pathways.
This work was supported by a Canadian Institutes for Health Research Catalyst grant (funding reference number: 134160) and the Royal’s University Medical Research Fund. The funders had no role in the review or approval of the manuscript for publication.
The research team would like to recognize the following psychiatric programs at the ROMHC: Mood and Anxiety, Geriatric Psychiatry, and Youth Psychiatry, as well as the service users who participated in our study.
SH conceived of the study and was the grant holder. SM took the lead in drafting of the manuscript, completed participant recruitment, and data collection activities. DJC conducted all statistical analyses. SL completed participant follow-up and data collection activities. JK and ZS completed the process evaluation of qualitative interviews. JK and KG conducted qualitative analysis of the process. All authors reviewed and approved the final manuscript.
Conflicts of Interest
CONSORT EHEALTH Checklist V1.6.PDF File (Adobe PDF File), 20847 KB
Supplemental file – Screenshots from The Journal.DOCX File , 489 KB
Mean and median scores on the EuroQol 5 dimension (3 levels) visual analog scale (n=94).DOCX File , 14 KB
- Wang PS, Aguilar-Gaxiola S, Alonso J, Angermeyer MC, Borges G, Bromet EJ, et al. Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. Lancet 2007 Sep 8;370(9590):841-850 [FREE Full text] [CrossRef] [Medline]
- Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med 2006 Nov;3(11):e442 [FREE Full text] [CrossRef] [Medline]
- Zalsman G, Hawton K, Wasserman D, van Heeringen K, Arensman E, Sarchiapone M, et al. Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry 2016 Jul;3(7):646-659. [CrossRef] [Medline]
- NICE. National Institute for Health and Care Excellence. London, UK: National Institute for Health and Care Excellence; 2009 Oct 28. Depression in Adults: Recognition and Management Management. Clinical Guideline URL: https://www.nice.org.uk/guidance/cg90/resources/depression-in-adults-recognition-and-management-pdf-975742636741 [accessed 2019-10-25]
- Stéphan F, Le Galudec M, Tritschler-Le Maitre M, Berrouiguet S, Walter M. Review of various cognitive behavioral therapy programs on the Internet for the treatment of depression. Article in French. Revue des différents programmes de thérapies cognitivo-comportementales sur Internet dans le traitement de la dépression. J Thérapie Comport Cogn 2014 Dec;24(4):138-143. [CrossRef]
- Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
- van Straten A, Cuijpers P, Smits N. Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J Med Internet Res 2008 Mar 25;10(1):e7 [FREE Full text] [CrossRef] [Medline]
- Donker T, Bennett K, Bennett A, Mackinnon A, van Straten A, Cuijpers P, et al. Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: randomized controlled noninferiority trial. J Med Internet Res 2013 May 13;15(5):e82 [FREE Full text] [CrossRef] [Medline]
- Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther 2018 Jan;47(1):1-18. [CrossRef] [Medline]
- Lewis CE, Farewell D, Groves V, Kitchiner NJ, Roberts NP, Vick T, et al. Internet-based guided self-help for posttraumatic stress disorder (PTSD): randomized controlled trial. Depress Anxiety 2017 Jun;34(6):555-565. [CrossRef] [Medline]
- Dear BF, Fogliati VJ, Fogliati R, Johnson B, Boyle O, Karin E, et al. Treating anxiety and depression in young adults: a randomised controlled trial comparing clinician-guided versus self-guided Internet-delivered cognitive behavioural therapy. Aust N Z J Psychiatry 2018 Jul;52(7):668-679. [CrossRef] [Medline]
- Topooco N, Berg M, Johansson S, Liljethörn L, Radvogin E, Vlaescu G, et al. Chat- and internet-based cognitive-behavioural therapy in treatment of adolescent depression: randomised controlled trial. BJPsych Open 2018 Jul;4(4):199-207 [FREE Full text] [CrossRef] [Medline]
- Rollman BL, Belnap BH, Abebe KZ, Spring MB, Rotondi AJ, Rothenberger SD, et al. Effectiveness of online collaborative care for treating mood and anxiety disorders in primary care: A randomized clinical trial. JAMA Psychiatry 2018 Jan 1;75(1):56-64 [FREE Full text] [CrossRef] [Medline]
- Arjadi R, Nauta MH, Scholte WF, Hollon SD, Chowdhary N, Suryani AO, et al. Internet-based behavioural activation with lay counsellor support versus online minimal psychoeducation without support for treatment of depression: a randomised controlled trial in Indonesia. Lancet Psychiatry 2018 Sep;5(9):707-716. [CrossRef] [Medline]
- Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs technician assistance. PLoS One 2010 Jun 8;5(6):e10939 [FREE Full text] [CrossRef] [Medline]
- Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011 Mar 10;13(1):e30 [FREE Full text] [CrossRef] [Medline]
- Thompson SC, Schlehofer MM. The many sides of control motivation: motives for high, low, and illusory control. In: Shah JY, Gardner WL, editors. Handbook of Motivation Science. New York, NY: The Guilford Press; 2008:41-56.
- Kleiboer A, Smit J, Bosmans J, Ruwaard J, Andersson G, Topooco N, et al. European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries. Trials 2016 Aug 3;17(1):387 [FREE Full text] [CrossRef] [Medline]
- Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010 Oct 13;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
- Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, REEACT Team. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. Br Med J 2015 Nov 11;351:h5627 [FREE Full text] [CrossRef] [Medline]
- Kenter RM, Cuijpers P, Beekman A, van Straten A. Effectiveness of a web-based guided self-help intervention for outpatients with a depressive disorder: Short-term results from a randomized controlled trial. J Med Internet Res 2016 Mar 31;18(3):e80 [FREE Full text] [CrossRef] [Medline]
- Hatcher S, Whittaker R, Patton M, Miles WS, Ralph N, Kercher K, et al. Web-based therapy plus support by a coach in depressed patients referred to secondary mental health care: randomized controlled trial. JMIR Ment Health 2018 Jan 23;5(1):e5 [FREE Full text] [CrossRef] [Medline]
- Depression and Anxiety. Wellington, NZ: New Zealand Health Promotion Agency; 2019. The Journal: Learn How to Feel Better with The Journal URL: https://depression.org.nz/get-better/the-journal/ [accessed 2019-10-25]
- Hatcher S, Sharon C, Parag V, Collins N. Problem-solving therapy for people who present to hospital with self-harm: Zelen randomised controlled trial. Br J Psychiatry 2011 Oct;199(4):310-316. [CrossRef] [Medline]
- Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weert HC. Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review. Gen Hosp Psychiatry 2007;29(5):388-395. [CrossRef] [Medline]
- Kroenke K, Spitzer RL. The PHQ-9: A new depression diagnostic and severity measure. Psychiatr Ann 2002 Sep 1;32(9):509-515. [CrossRef]
- Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
- Arroll B, Goodyear-Smith F, Crengle S, Gunn J, Kerse N, Fishman T, et al. Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Ann Fam Med 2010;8(4):348-353 [FREE Full text] [CrossRef] [Medline]
- Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med 2001 Jul;33(5):337-343. [CrossRef] [Medline]
- Hurst NP, Kind P, Ruta D, Hunter M, Stubbings A. Measuring health-related quality of life in rheumatoid arthritis: validity, responsiveness and reliability of EuroQol (EQ-5D). Br J Rheumatol 1997 May;36(5):551-559. [CrossRef] [Medline]
- McMillan D, Gilbody S, Richards D. Defining successful treatment outcome in depression using the PHQ-9: a comparison of methods. J Affect Disord 2010 Dec;127(1-3):122-129. [CrossRef] [Medline]
- Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J 2015 Mar 19;350:h1258 [FREE Full text] [CrossRef] [Medline]
- Canadian Internet Registration Authority. Ottawa, ON: Canadian Internet Registration Authority; 2019. Canada's Internet Factbook URL: https://cira.ca/factbook/canada%E2%80%99s-internet-factbook-2018 [accessed 2019-07-03]
- Statistics Canada. Ottawa, ON: Statistics Canada; 2017. Census Profile, 2016 Census URL: https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/prof/details/page.cfm?Lang=E&Geo1=CMACA&Code1=505&Geo2=PR&Code2=35&SearchText=Ottawa+-+Gatineau&SearchType=Begins&SearchPR= 01&B1=Education&TABID=1&type=0 [accessed 2019-07-03]
|CBT: Cognitive behavioral therapy|
|EMR: electronic medical record|
|EQ-5D-3L: EuroQol 5 Dimensions (3 levels)|
|EQ-5D-VAS: EuroQol 5 dimensions visual analogue scale|
|MD: mean difference|
|NICE: National Institute for Health and Care Excellence|
|PHQ-9 Q9: Patient Health Questionnaire question 9|
|PHQ-9: Patient Health Questionnaire|
|RCT: randomized controlled trial.|
|REEACT: Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy|
|ROMHC: Royal Ottawa Mental Health Centre.|
Edited by G Eysenbach; submitted 05.07.19; peer-reviewed by K Stasiak, D Leightley, L Saslow; comments to author 30.08.19; revised version received 25.10.19; accepted 17.12.19; published 09.06.20Copyright
©Sarah MacLean, Daniel J. Corsi, Sadie Litchfield, Julia Kucharski, Kira Genise, Zeynep Selaman, Valerie Testa, Simon Hatcher. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.06.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.