Depression is a prevalent and disabling disorder with a high risk of relapse and large individual and societal costs [1-6]. Effective treatments do exist [7], although a large gap is seen between the need for and use of treatments [8]. This gap has led researchers to explore alternative modes of treatment delivery. One such novel treatment format is internet-based cognitive behavioral therapy (iCBT) [9-17], in which the patient is administered access to an online treatment program based on CBT. The highest clinical effect is seen when clinical guidance is provided during the course of treatment [18-20]. However, despite the evidence for the effect of guided iCBT, there is a need for increased therapist contact among some patient groups as well as a need to provide a treatment format, which is more compatible with, and thus easier to implement in, the existing health care services [21-24].

Combining iCBT with traditional face-to-face (FtF) consultations in a blended CBT format (B-CBT), in which both online components and FtF sessions are included in 1 coherent CBT protocol, may alleviate some of the difficulties associated with iCBT for depression, while preserving some of the advantages of both iCBT and FtF-CBT alike. First, by including FtF sessions, the therapist can individualize the therapy taking the idiosyncratic case formulation of the patient, the specific disorder, and possible comorbidity into account. Second, as B-CBT in the format tested in this study only provides half the number of sessions as traditional FtF-CBT, the capacity of the treating clinician is increased compared with traditional CBT. Third, the burden and cost of travel by the patient can be reduced compared with FtF-CBT. Fourth, the online modules are available at the time and place needed by the patients—and they can be re-viewed multiple times. Fifth, the inherently structured format of the online modules ensures high treatment fidelity, for example, by delivering the same psychoeducation and exercises to all patients. Sixth, one of the principal barriers for the uptake of iCBT seems to be skepticism concerning allotting the majority of therapy to a computer [25], a barrier possibly alleviated by the B-CBT [26]. Finally, the blended format is more compatible with the existing health care services and as a consequence should be easier to implement than iCBT [27].

Few studies have investigated the use of blended care combining internet-based psychotherapeutic modules and FtF sessions into 1 coherent treatment manual to treat adult depression [28-31]. Generally, however, they do indicate positive outcomes. In a randomized controlled trial conducted in primary care in Tromsø, Norway, clinical psychologists delivered 30-minute sessions following each online module [29]. They were able to document a significant difference with a moderate to large effect size (d=0.65) on depressive symptoms (Beck Depression Inventory II [BDI-II]) favoring blended care over waiting list. The intervention predominantly received positive evaluations suggesting acceptability and satisfaction with the treatment. In addition, a qualitative study found that the FtF consultations increased motivation to persist with the iCBT program [32]. Another recent example is the development and initial evaluation of a program for B-CBT in The Netherlands. This was tested at an outpatient clinic of a specialized mental health care center in Amsterdam. The study was designed as a feasibility study and included only 9 patients. However, the patients perceived the intervention as positive, although the authors rightly noted that no conclusion can be derived from such a small sample [31]. A cohort study from the United States found a significant and large reduction in symptoms of depression but had no comparison group [33].

In this study we compared directly the clinical effect on adult depression of B-CBT and FtF-CBT in a randomized, controlled, noninferiority study in parallel groups, recruiting from a routine care iCBT clinic in the Region of Southern Denmark.

The primary aim of this study was to compare the clinical effectiveness of B-CBT for major depressive disorder in adults with treatment as usual (TAU) defined as 12 sessions of FtF-CBT. It is hypothesized that B-CBT will be no less clinically effective than FtF-CBT, and that it will be acceptable and satisfactory to patients and clinicians.

The study was a randomized, controlled, noninferiority trial comparing B-CBT with FtF-CBT. It was part of the research program e-Mental Health Research (ENTER) located in and coordinated from the Centre for Telepsychiatry in the Mental Health Services of Southern Denmark, Odense. Additionally, this study was affiliated with the European Union (EU) study E-COMPARED [34]. However, the E-COMPARED study ended prior to this, and thus does not include the total sample. In this article we explore the full data set of the trial.

The trial was approved by the Ethics Committee of the Region of Southern Denmark (registration number S-20150150) prior to instigation. The trial followed the Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects [35]. All participants received both written and oral information about the trial and signed written informed consent before entering the study.

The trial was registered with ClinicalTrials.gov NCT02796573. The trial protocol was published previously [36].

Funding was granted from the Research Fund of the Mental Health Services of Southern Denmark, and from the Innovation Fund Denmark, as part of the project ENTER (ID: 5159-00002B). Both are public funds. None of the funds have had any role in the design of the study nor in the collection, analysis or interpretation of the data, or writing of the manuscript.

An independent researcher from the EU study E-COMPARED [34], who was not involved in the trial, performed the randomization at an individual level, stratified by country after eligibility and baseline measurement. A random number generator (Random Allocation Software) was applied with an allocation ratio of 1:1. Block randomization was used with block sizes varying from 8 to 14 allocations per block.

It was not possible to blind the patients nor the treating clinicians to the allocated treatment. However, those assessing the participants were blinded to allocation as were the researchers and statisticians involved up until the point of interpretation of the results. Some questionnaires were only administered to the B-CBT group and were kept in a separate data set.

After consent was granted, baseline measures were administered prior to randomization. Follow-up measurements were conducted 3, 6, and 12 months after baseline. Additionally, weekly measures were provided during treatment. The questionnaire packages were administered online using a secure web application for building and managing online surveys (REDCap), except for the weekly monitoring of the B-CBT group, for which the packages were administered automatically by the treatment program.

Data were stored by the Odense Patient data Exploratory Network (OPEN) [45]. Data were collected, transferred, and stored securely electronically as approved by the Danish Data Protection Agency (journal number: 14/26634, registration number: 2008-58-0035).

The PHQ-9 [39] was used as the primary outcome measure. The PHQ-9 is a 9-item questionnaire developed to measure depressive symptomatology in the primary health care sector. The 9 items are each scored on a 0-3-point scale with the total score ranging from 0 to 27, with higher scores indicating more severe depression. The authors suggest using cut-off points of 5, 10, and 15 for mild, moderate, and severe levels of depression, respectively, in the guide to the instrument substantiated by a review [40]. The PHQ-9 has been shown to have good psychometric properties [46].

A number of additional measures were administered to assess different aspects of the participants’ symptomatology and experience during the treatments. The 16-item Quick Inventory of Depressive Symptomatology Self-Report (QIDS-16-SR) [47,48] was used in addition to the PHQ-9 because it is a promising questionnaire for assessing depressive symptoms, especially in specialized mental health care and to conduct secondary analyses of primary latent construct of interest: depression. To measure the participants’ satisfaction with the treatments, the 8-item Client Satisfaction Questionnaire (CSQ-8) [49,50] was used. The Credibility and Expectancy Questionnaire (CEQ) [51] was used to measure the participants’ expectancy and judgment of credibility of the treatments. Finally, the level of therapeutic alliance was measured using the Working Alliance Inventory-Short Revised (WAI-SR) [52-54] and was rated by both the participants and the clinicians. For further description of the measures used, we refer to Mathiasen et al [36].

Table 2 shows the baseline characteristics of all participants and Figure 1 shows the patient flow. In total, 76 were randomized, with 38 allocated to each group. Attrition was somewhat unevenly distributed between groups with 8 being lost to follow-up in the FtF-CBT group and 16 in the B-CBT group. Nonetheless, due to the weekly measurement scheme and the use of mixed effects models, all but one was included in the primary analyses.

The included sample was predominantly female (56/76, 74%) and young with a mean age of 35.0 (SD 13.96) years (median 30 years), although a large age range was seen (18-71 years). Most had moderate to highly severe levels of depression (66/76, 87%) with a mean score of 15.25 (SD 4.04) on the PHQ-9.

No significant differences were observed between the 2 groups on baseline characteristics except for scores on the CEQ measuring the participants’ expectations and credibility of the treatments (see Table 2 for P values). The participants in the B-CBT group scored lower on treatment credibility and expectancy of treatment outcome. This raised suspicion as to whether the difference could have been caused by the participants being aware of their group allocation prior to responding to the questionnaire. However, when investigated, it did not seem to be the case. Likewise, no obvious outliers were driving the difference and the distribution of scores seemed reasonable upon visual inspection. A sensitivity analysis of the primary analysis was conducted controlling for the credibility and expectancy scores, but it did not change the outcome.

Among the included sample, 7 were on the brink of violating exclusion criteria, 3 were in psychological treatment at the point of assessment, 2 had some obsessive compulsive disorder symptoms, and 2 were not depressed according to MINI, but scored 9 and 17 on the PHQ-9, respectively. When comparing analyses including or excluding these cases, the outcome did not change. To avoid causing any changes to the analysis plan, all analyses were performed including these participants.

The main aim of this study was to compare the clinical effectiveness of B-CBT with traditional FtF-CBT, because the blended format may hold the promise to combine advantages of the traditional and the new format of delivery. In this study, we found very similar trajectories of improvement in both groups as well as on measures of other parameters, such as working alliance and retention. However, it was possible to detect a significant difference between groups in slight favor of FtF-CBT.

The sample corresponded well with what is seen among patients with depression in the primary health care sector in Denmark regarding gender and age distribution [3,62]. Further, the distribution of the highest education level resembles that of the general Danish population [63].

The mean symptomatic change observed in the B-CBT condition closely approximated that of the FtF-CBT group. This is in line with meta-analyses of guided iCBT for depression [16,64] and exceeded what has been observed at the clinic “Internetpsykiatrien,” from which this study recruited [17]. It also aligned well with the large effect (d=1.08) seen in the study of B-CBT by Lungu et al [33]. Nonetheless, an interaction effect could be detected between the 2 groups favoring FtF-CBT, although the effect was very small. This effect seems to be driven by the FtF group experiencing a faster symptom reduction during treatment, an effect that disappeared at 6 months’ follow-up. However, at 12 months’ follow-up, the FtF group showed a larger reduction compared with B-CBT. It is important to note, though, that none of these differences were statistically significant, thus a difference between groups cannot be inferred. Unfortunately, with the variance observed, it is not possible to infer noninferiority either (a difference no bigger than d=.02), due to the CIs stretching beyond the noninferiority margin. Consequently, although promising, the study is inconclusive regarding noninferiority but may support no superiority of either treatment.

The working alliance between the patient and clinician has often been argued to be one of the most important nonspecific factors of psychotherapy [65]. It is, therefore, very interesting that although half of the sessions in B-CBT were computerized, the therapeutic alliance was rated equally well in both groups. There was a tendency among the clinicians to rate it higher in the FtF group, but the difference was not significant. Similar findings are also emerging in other studies of blended care [66]. Furthermore, these studies are starting to point to details differing between B-CBT and FtF-CBT, for example, therapist ratings sometimes correlating more with treatment outcome than patient ratings conflicting with research on the working alliance in FtF-CBT [65]. Further study detailing the dynamics of the therapeutic alliance in B-CBT and the difference between that and FtF-CBT is needed.

Acceptability of the blended format seemed to be high as judged by levels of client satisfaction and working alliance, where no significant differences were observed. Furthermore, high retention rates among participants and high treatment fidelity rates for the clinicians indicated satisfaction and acceptability with the treatment.

As is commonly found [67-69], a negative correlation between intercept and slope was seen, indicating that a higher baseline severity of depression was associated with larger symptom reduction.

Interestingly, 1 variable was able to distinguish between the 2 groups in predicting outcome differently. An interaction effect was observed between employment status and group. Being part-time employed or unemployed both favored the blended care group. We speculate that this may be due to the B-CBT treatment always being available, possibly increasing the chance of treatment engagement if the participant has more free time. Consequently, this is a potential candidate variable for stratification of treatment or a prescriptive variable. In a different approach comparing variables predicting outcomes separately for the 2 groups in a larger sample across 4 countries [70], a lower quality of life and being widowed predicted lower treatment outcome in the blended condition. However, this approach does not include the parameters in a single model comparing the conditions directly in an interaction term. Nonetheless, these findings are encouraging and prompt the need to further study potential variables for stratification of patients.

This study is well aligned with previous observations of completion rates in both guided iCBT and traditional CBT [71-74]. The reasons provided by either therapists or patients for treatment dropout varied and there were too few to differentiate between the groups.

In disagreement with what has previously been found in guided iCBT, we observed that no baseline characteristics predicted noncompletion [25,75,76]. It may be speculated to be caused by the increased therapist contact in the blended format, which may serve as a protective factor against noncompletion.

This study compared directly the formats of delivery with a minimum of the variance explained by differences in therapeutic methods, which is both a strength and a weakness of the design. While it lends itself well to compare the 2 treatment formats, it also somewhat limits the ecological validity, making it more difficult to make inferences about the clinical effect in routine care. Furthermore, because the study recruited from Internetpsykiatrien, which offers self-referral, even though the clinic is situated in secondary care, it can be difficult to generalize to future implementations. Additionally, only the B-CBT group received reminders about homework assignments. This might be a confounder, for example, there is a risk participants in this group grew weary of the reminders, thus affecting the perception of the treatment negatively. Finally, due to the small sample size, we had difficulty inferring noninferiority, although the many observations and advanced statistical procedures appear to have compensated for that to some degree. The large EU study E-COMPARED will be able to pool data from many studies, including this one, and may thus be able to reach more robust conclusions about noninferiority.

In this study, feasibility of B-CBT was demonstrated as well as large and significant within-group effect sizes were produced. In fact, it was seen that practically without loss of treatment effect, completion rates, and therapeutic alliance, it was possible to substitute half of the FtF consultations with online modules when treating adult depression. This is remarkable and lends support to the hypothesis of noninferiority of B-CBT and should lead to the further study of this promising treatment format. However, it should also be noted that small differences were observed favoring the FtF-CBT group. Although not significant, it may be that FtF treatment works faster, and has a better long-term effect for some patients. The results, therefore, need to be replicated in larger samples or with pooled data from multicenter trials as will be done in the E-COMPARED study. Additionally, further studies should explore the applicability of B-CBT in different patient populations and clinical settings. Furthermore, participants’ digital health literacy should be measured in future studies.