Depression is a common mental disorder [1] with a high social burden [2] and significant impact on suicidality [3] and quality of life. Most treatment for depression occurs in primary care. In secondary care, treatment is often limited to drug therapies, in part, due to long waiting lists to see psychological therapists face-to-face. This is contrary to recommendations about the importance of nondrug therapies by the National Institute for Health and Care Excellence (NICE), based in the United Kingdom, and other institutions [4]. In secondary and tertiary mental health care centers in Ontario, at the time of this trial, the waiting time to be seen by a psychiatrist for depression was between 9 months and 1 year.

There is evidence that web-based therapies can reduce the symptoms of depression [5]. Randomized controlled trials (RCTs) have demonstrated the effectiveness of web-based cognitive behavior therapy (CBT) [6,7], problem-solving therapy [8], interpersonal therapy [9], and psychodynamic therapy. Computerized CBT is also recommended in the NICE guidelines for the treatment of mild to moderate depression [4]. However, most of the RCTs of web-based therapies have been conducted in community samples, often recruited from the internet. These populations are self-selecting and, although their scores on depression rating scales may be comparable with clinical populations, they often differ in terms of comorbidity, duration of symptoms, and impact on daily functioning.

The delivery of web-based therapy as a treatment for depression can be performed in one of two ways: through the use of supports to assist patients through the web-based therapy (guided model) or through the self-help use of computerized treatment (unguided model). In the guided model, patients are provided support as they progress through web-based therapy. In some cases, highly trained clinicians have been used to fill this role [10]; however, their advanced training is costly and does not appear to provide any added benefit. For instance, despite the use of a clinician in the delivery of web-based therapy for the treatment of depression and anxiety among young adults, Dear et al [11] did not find any significant differences in symptomatology when compared with the unguided condition. One solution that has been implemented to mitigate this issue is the use of coaches to support patients progressing through web-based therapy, answering both technical questions about how the program works as well as providing support and encouragement. To date, coaches have included both students [12] and licensed professionals from a variety of backgrounds, including psychology [13], counseling [14], and social work [15,16]. The components of effective coaching are still uncertain [17], with most of the literature emphasizing the technical aspects of an internet intervention. It is not clear whether the professional background of the coach, the frequency of contact, or the content of the coaching sessions affects treatment outcomes.

Several systematic reviews have demonstrated that internet-based interventions for depression have effect sizes that are comparable with face-to-face interventions, whereas unguided interventions have smaller effect sizes [7,18,19]. However, the evidence is inconsistent, with head-to-head comparisons of guided versus unguided interventions showing mixed results. The largest study of guided web-based therapy compared with usual care has been the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) study [20], which was a UK-based RCT of 691 patients with depression in primary care. Participants were randomized to one of three treatment groups: (1) usual care; (2) Beating the Blues, a guided commercial web-based program; or (3) MoodGYM, a guided free web-based program. The study found no difference in depression outcomes between the three groups. Similarly, Kenter et al [21] compared student-assisted internet-based problem-solving therapy with an information-only waitlist control and found minimal differences in depression severity (Cohen d=0.07), as measured by the Centre for Epidemiology Studies Depression scale.

Web-based interventions for depression struggle with engagement and adherence to treatment protocols. For example, in the Beating the Blues and MoodGYM arms of the REEACT trial, participants completed only a median of 1 to 2 sessions and received only 6 min of technical support time, 5 emails, and almost no text messages from the telephone support workers. Further, approximately 1 in 5 participants randomized to either of the web-based therapy conditions did not access the programs at all. Similar problems were encountered in the study conducted by Kenter et al [21], with only 36% of participants receiving an adequate dosage of the intervention, defined by the authors as 4 of 5 lessons of web-based therapy. From a public health perspective, low adherence to web-based therapies dilutes their effectiveness. However, it is unclear if, for individual patients, adhering to a complete course of web-based therapy is better than control treatments.

We have previously reported on an RCT of The Journal comparing guided web-based therapy with an information-only waitlist control in patients with depression referred to secondary mental health services in New Zealand [22]. Participants were recruited face-to-face during their triage visit at community mental health centers, and the study found no difference in depression or service use after 12 weeks. However, all participants in this study were also receiving mental health care from clinicians, which may have outweighed any effect of the web-based therapy. In this study, we report on an RCT of The Journal [23], facilitated by a coach, compared with an information-only waitlist control group for the treatment of depression in patients referred to secondary mental health services in Canada. In this setting, access is a significant issue, with patients often waiting for over a year to access mental health services. During this time, patients on the waiting list do not receive any other mental health care other than routine follow-up from their family physician; therefore, the provision of web-based therapy could potentially be used as an alternative to referral to specialized mental health services. This study gave us an opportunity to refine the study intervention. In contrast to the New Zealand study, we opted to use an information-only control group as it is a low-cost, low-risk alternative that could easily be implemented in clinical practice. Similarly, we were able to refine the coaching aspect of the intervention. We hypothesized that, after 12 weeks of treatment, participants receiving coach-guided web-based therapy would experience a greater reduction in depressive symptoms and health service use.

The design of the trial was an RCT with two parallel groups. This trial has been reported according to the Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth (CONSORT-EHEALTH; Multimedia Appendix 1).

Potential participants were patients referred to the Royal Ottawa Mental Health Centre (ROMHC, Ottawa, Canada) with symptoms of depression or dysthymia who were on a waiting list for treatment in the following psychiatric programs: Mood and Anxiety, Geriatric Psychiatry and Youth Psychiatry. The ROMHC has 284 inpatient beds and acts as a specialized mental health facility for residents of communities across Eastern Ontario. These programs are aimed at treating people with complex and serious mental illnesses that are often resistant to treatment. Patients are referred to treatment in these programs directly by their family physicians. The first appointment in the programs is with a psychiatrist who then decides on a treatment plan with the patient. At the time of this clinical trial, the waitlist for the Mood and Anxiety Program was between 9 months and 1 year.

Eligibility criteria for participation in the trial are outlined in Textbox 1. Potentially eligible patients were contacted using the following methods: patients who had completed a consent to be contacted for research as part of their referral documentation were contacted by telephone. Those who did not complete the consent to be contacted for the research section of the referral were contacted by mail, as per institutional policies. Interested patients were preliminarily screened by a research assistant for eligibility to participate in the study. Eligible patients were then asked to attend a face-to-face consenting appointment with a research assistant. A minority of patients who could not travel to the clinic were asked to consent via mail (n=3).

Based on the previous studies that used the PHQ-9 as their primary outcome measure, we expected the mean pretreatment score to be 17.0 (SD 4.0). To detect a difference in PHQ-9 scores between the two groups of at least 3 points, an established minimal clinically important difference [31], we would need 44 participants in each group with a 2-sided alpha of .05, and a power of 80.0% (effect size of 0.6). Allowing for a 25.0% dropout rate, we aimed to recruit a total of 110 participants.

Randomization was completed by the Ottawa Methods Centre at the Ottawa Hospital Research Institute, with allocations kept in sequential sealed envelopes at the study base. Participants were randomized in a 1:1 allocation, and there were no restrictions. After providing consent, participants were randomized by a research assistant according to the allocation in the sealed envelopes.

Owing to the nature of the intervention, neither participants nor study staff were blinded to the treatment allocation. Outcome assessments were collected by delegated study staff who were not blinded to the treatment allocation.

Group differences in demographic and pretreatment measures were analyzed using independent samples t tests. Changes in participants’ scores from pretreatment to follow-up at 12 weeks were assessed using repeated measures analysis of variance with mixed linear modeling to account for missing variables. This model included the following variables: treatment group (control or intervention), PHQ-9 scores at 4 different time points (baseline, week 2, week 6, and week 12), gender (male or female), and age. PHQ-9 scores were entered as within-subject variables, treatment group as between-subject factors, and age and gender as covariates. Statistical analyses were conducted using the general linear modeling repeated measures procedure in IBM SPSS Statistics 25 for Windows. To assess significant differences in outcome measures, the last observations for the PHQ-9, EQ-5D-3L and EQ-5D-VAS were carried forward for the 6- and 12-week time points.

Differences in suicidal thoughts between the two groups were assessed using an independent samples t test. Differences in proportions of service use were assessed using chi-square tests. In addition, we also assessed whether participants experienced a clinically important reduction in depression symptoms defined as a PHQ-9 score of 9 or less or a 50.0% reduction in scores [31]. This was described using percentages and frequencies, and chi-square was used to assess the significance of the differences in proportions. Assessments of normality were completed using the Shapiro-Wilk test for continuous data, and the Mann-Whitney U test for nonparametric data.

As per the recommendations outlined in the Medical Research Council’s guidelines for the assessment of complex interventions [32], we conducted a process evaluation to assess the context, implementation, and mechanisms of impact of the study intervention. The process evaluation outcome measures are outlined in Table 3. We also conducted semistructured interviews with participants within 6 months of study completion to assess their experience of the study intervention, including access to and functionality of The Journal, the therapeutic content and value of The Journal, and the experience of working with a coach. Interviewees were identified using a purposive sampling approach and were stratified to include participants in both arms of the trial and varying levels of engagement (eg, not engaged, moderately engaged, and highly engaged). Interviews were conducted until data saturation was reached (n=15).

Individual interviews were conducted by two female University of Ottawa Department of Psychiatry residents (JK and ZS) who were independent of the research team that conducted the RCT. Following transcription, two independent coders (JK and KG) analyzed the material using a thematic, grounded theory approach. The coding took place in two stages, with coders meeting during the first stage to discuss emergent codes, reconcile definitions, and compare coding rationales. The two coders had not been involved in delivering the treatment or any of the previous study contacts.

The study received approval from the Royal Research Ethics Board (Protocol 2014001). All participants provided informed consent before participation in both the RCT and qualitative interviews.

Recruitment for this study took place over 11 months, from May 2015 to April 2016, and a total of 1316 patients were preliminarily screened for eligibility by examining their referral documentation (Figure 1). Of these, 45.9% (605/1316) could not be reached to complete a full screening for eligibility; 40.4% (532/1316) could be contacted were not eligible to participate, and 13.6% (179/1316) were eligible and could be contacted.

Of the 45.9% (605/1316) of patients who could not be contacted to complete a full eligibility screen, 25.4% (154/605) provided consent to be approached about research studies but did not reply to telephone or mail invitations; 63.1% (382/605) were invited to discuss the study by mail with no response, and 11.3% (69/606) had incomplete mailing information listed on their referral documents.

Of the 40.4% (532/1316) of patients who could be contacted but were ineligible to participate, 84.9% (452/532) were referred for a reason other than depressive symptoms; 5.6% (30/532) had cognitive impairments rendering them unable to use a computer; 4.5% (24/532) were no longer on the waitlist at the time of screening; 2.6% (14/532) were under 16 years of age at the time of screening; 1.12% (6/532) were unable to read and write in English; and, 1.12% (6/532) did not have a valid Ontario Health Insurance Plan number.

Of the 13.6% (179/1316) of patients who were eligible to participate, 46.9% (84/179) declined to participate, and 53.1% (95/179) consented to participate in the study. Table 4 lists the reasons for patients declining to participate.

The majority of participants enrolled in the study were recruited from the Mood and Anxiety Outpatient Clinic (n=92), and 3 were recruited from the Geriatric Psychiatry Program at the ROMHC. No participants were enrolled from the Youth Psychiatry Program. Table 5 describes the demographic characteristics of the participants. There were significantly more women in the intervention group and more men in the control group (Χ²₁=6.6; P=.01).

Participants in the intervention group completed a mean number of 5.0 (SD 2.3) lessons in The Journal and 8.8 (SD 3.1) sessions with the coach. In the control group, 10% (5/48) of participants reported accessing The Journal. In the control group, 17% (8/48) of participants scored 9 or below on the PHQ-9 at baseline compared with 23% (11/47) of participants in the intervention group. Owing to gender imbalances between the groups, we also conducted a post hoc gender analysis of changes in PHQ-9 scores and health service use.

The mean PHQ-9 score was lower in the intervention group than in the control group at all study time points (Table 6). The proportions of missing values were comparable between the 2 groups at all assessment points, with the exception of week 12 (2/46 , 4% in the control group compared with 0/47, 0% in the intervention group). The Shapiro-Wilk test of normality revealed that all data were normally distributed with the exception of the control group at the week 12 study time point (W₄₆=0.95; P=.04).

At 12 weeks, the mean PHQ-9 score in the intervention group was 11.3 (SD 6.4) and was 12.4 (SD 6.4) in the control group (t₈₆=0.76; P=.45, 95% CI −1.9 to 3.8). To account for the non-normality in scores at week 12, mean differences (MDs) at this time were also assessed using a Mann-Whitney U test of significance, which confirmed that the difference between the two groups was not statistically significant (Mann-Whitney U statistic=862.0; Z=−0.85; P=.40). The mean change in PHQ-9 scores from baseline to week 12 was −3.6 (SD 6.6) in the intervention group and −3.1 (SD 6.2) in the control group, which was not statistically significant (t₉₁=−0.37; P=.72, 95% CI −3.1 to 2.2).

Excluding the 10% (5/48) of participants in the control group who also accessed The Journal did not result in any statistically significant differences in mean PHQ-9 scores compared with the intervention group. The mean score at week 12 for the control group excluding those who accessed The Journal (n=41) was 12.4 (SD 6.2) compared with 10.7 in the intervention group (t₈₆=0.76; P=.45, 95% CI −0.9 to 4.3).

The repeated measures modeling found that scores on the PHQ-9 differed significantly by study time point, irrespective of group allocation (F_2.6,222.8=11.59, P<.001). However, this relationship was not sustained once treatment group was taken into account (F_1,87=1.46, P=.23). Similarly, modeling found that scores on the PHQ-9 were not significantly associated with gender (F_1,87=0.33, P=.57) or the interaction between treatment group and PHQ-9 scores (F_1,87=0.70; P=.53). However, results were significantly associated with age of participants (F_1,87=5.97, P=.02, ŋ²=0.06).

Among patients on a waiting list referred to secondary care for depressive symptoms, a trial of usual care plus coach-facilitated web-based therapy compared with usual care plus information-only waitlist control group found no statistically significant differences in mean depression scores after 12 weeks. However, participants who completed 6 or more lessons in The Journal reported significantly greater reductions in symptoms compared with controls. Participants in the intervention group also reported better quality of life after 12 weeks of treatment. There was no difference in service use between the two groups. During the qualitative interviews conducted as part of the process evaluation, participants identified the role of the coach as a major factor in their completion of the web-based therapy program.

The major strength of this study is that it is the first study of coach-assisted web-based therapy in a secondary care setting in Canada. This study demonstrated that it is possible to incorporate the use of a coach within this clinical setting. However, due to limitations imposed by institutional policies, study staff were restricted in their ability to recruit participants, with only 12% (95/784) of potentially eligible patients agreeing to participate in the trial. It is reasonable to expect greater reach if this intervention was rolled out by a clinical service that could approach people directly at the time of referral.

There was also a possibility of contamination between the arms of the trial. Given that The Journal is a publicly accessible website, any participant was able to access it, and there was little that could be done by the study team to prevent this. However, our experience is that patients in secondary care do not widely use The Journal, with only 10% (5/48) participants in the control group reporting use of The Journal during the treatment period. Furthermore, contamination is likely to bias the study to showing no difference (as the control group could use The Journal unguided), so any differences that are found are likely to be more believable. Finally, the ability to generalize study findings to the larger Canadian population may be limited as a result of our highly educated population.

Some participants who were interviewed as part of the process evaluation expressed issues with recall given that the interviews took place 6 months after study completion. It is possible that interviews conducted closer to study completion would have yielded richer qualitative data.

A key contribution of this study is that participant engagement with and adherence to the intervention was much higher than that reported in previous work, with 62% (29/47) of participants in the intervention group receiving a full dose of treatment, completing an average of 5 out of 6 lessons in The Journal and 8.8 telephone coaching sessions with the coach. Comparatively, in the REEACT trial, participants only completed a median of 2 sessions of web-based therapy. Similarly, in the study by Kenter et al [21], only 36% of participants received the required 4 to 5 sessions of internet-based psychotherapy. This was found in spite of difficulties with ease of access, as The Journal was not available on mobile or tablet devices at the time of the study. This trial also highlights the importance of the uptake of the intervention to clinical outcomes, with participants who completed at least 6 lessons in The Journal experiencing a significantly greater reduction in depression symptoms than those who did not.

The previous trial of a guided version of The Journal in New Zealand secondary care showed no differences in clinical outcomes or service use. However, the patients in the New Zealand study had higher baseline depression scores, a mean PHQ-9 of 17.0, and were less engaged with the study intervention, completing only a mean of 2.5 lessons in The Journal. The magnitude of change in PHQ-9 scores was also different, with an intervention group change in PHQ-9 scores of 3.5 in Canada compared with 9.4 in New Zealand. This presumably reflects differences in usual care, with participants in New Zealand receiving mental health care, whereas Canadian participants received care from their primary care physicians while on a waiting list.

Web-based therapies are often promoted as a way to address long waiting lists for mental health care services. However, in this study, even though participants in the intervention group with more exposure to the study intervention were more likely to experience a significant reduction in symptoms, this had little impact on subsequent service use. Web-based therapies are part of a complex socio-technological system and, as such, cannot exist in a vacuum. To achieve improvement in patient outcomes, they must be integrated into a larger system of care. This could be achieved with a more explicit stepped care system supported by web-based therapies, with a more flexible response from providers based on patient need.

This study provides limited support for the potential use of web-based therapies within a stepped-care approach to the treatment of depression. However, an RCT is needed to determine the effectiveness of such an approach. The impact of providing digital services to those in greatest need, who are also the least likely to have access to high-speed connections, also needs to be taken into account. Future research on internet-based psychotherapy for depression needs to include the system of care in which it is used. This can be done through implementation science tools, which not only evaluate the effectiveness of web-based therapies but also the factors central to their uptake, such as reach, adoption, and sustainability.

The results of this study demonstrated that the use of guided web-based therapy for the treatment of depression is not more effective than information-only waitlist control. However, it showed that coach-guided web-based therapy has the potential to increase adherence and engagement with depression treatment protocols. Greater adherence resulted in greater effectiveness. More research is needed on the human component of coaching in conjunction with web-based therapy to examine what factors lead to greater adherence. Researchers also need to consider when and how web-based therapies should be integrated into existing clinical pathways.