Published on in Vol 15, No 8 (2013): August

The Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist Support for Depressed Primary Care Patients: Randomized Controlled Trial

The Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist Support for Depressed Primary Care Patients: Randomized Controlled Trial

The Clinical Effectiveness of Web-Based Cognitive Behavioral Therapy With Face-to-Face Therapist Support for Depressed Primary Care Patients: Randomized Controlled Trial

Original Paper

1Department of Psychology, Faculty of Health Sciences, University of Tromsø, Tromsø, Norway

2Centre for Mental Health Research, The Australian National University, Canberra, Australia

3Department of Community Medicine, Faculty of Health Sciences, University of Tromsø, Tromsø, Norway

4Department of Community Medicine, General Practice Research Unit, Faculty of Health Sciences, University of Tromsø, Tromsø, Norway

Corresponding Author:

Ragnhild Sørensen Høifødt, PsyD

Department of Psychology

Faculty of Health Sciences

University of Tromsø

Tromsø, 9037


Phone: 47 776 49230

Fax:47 776 45291


Background: Most patients with mild to moderate depression receive treatment in primary care, but despite guideline recommendations, structured psychological interventions are infrequently delivered. Research supports the effectiveness of Internet-based treatment for depression; however, few trials have studied the effect of the MoodGYM program plus therapist support. The use of such interventions could improve the delivery of treatment in primary care.

Objective: To evaluate the effectiveness and acceptability of a guided Web-based intervention for mild to moderate depression, which could be suitable for implementation in general practice.

Methods: Participants (N=106) aged between 18 and 65 years were recruited from primary care and randomly allocated to a treatment condition comprising 6 weeks of therapist-assisted Web-based cognitive behavioral therapy (CBT), or to a 6-week delayed treatment condition. The intervention included the Norwegian version of the MoodGYM program, brief face-to-face support from a psychologist, and reminder emails. The primary outcome measure, depression symptoms, was measured by the Beck Depression Inventory-II (BDI-II). Secondary outcome measures included the Beck Anxiety Inventory (BAI), the Hospital Anxiety and Depression Scale (HADS), the Satisfaction with Life Scale (SWLS), and the EuroQol Group 5-Dimension Self-Report Questionnaire (EQ-5D). All outcomes were based on self-report and were assessed at baseline, postintervention, and at 6-month follow-up.

Results: Postintervention measures were completed by 37 (71%) and 47 (87%) of the 52 participants in the intervention and 54 participants in the delayed treatment group, respectively. Linear mixed-models analyses revealed a significant difference in time trends between the groups for the BDI-II, (P=.002), for HADS depression and anxiety subscales (P<.001 and P=.001, respectively), and for the SWLS (P<.001). No differential group effects were found for the BAI and the EQ-5D. In comparison to the control group, significantly more participants in the intervention group experienced recovery from depression as measured by the BDI-II. Of the 52 participants in the treatment program, 31 (60%) adhered to the program, and overall treatment satisfaction was high. The reduction of depression and anxiety symptoms was largely maintained at 6-month follow-up, and positive gains in life satisfaction were partly maintained.

Conclusions: The intervention combining MoodGYM and brief therapist support can be an effective treatment of depression in a sample of primary care patients. The intervention alleviates depressive symptoms and has a significant positive effect on anxiety symptoms and satisfaction with life. Moderate rates of nonadherence and predominately positive evaluations of the treatment also indicate the acceptability of the intervention. The intervention could potentially be used in a stepped-care approach, but remains to be tested in regular primary health care.

Trial Registration: Australian New Zealand Clinical Trials Registry: ACTRN12610000257066; (Archived by WebCite at

J Med Internet Res 2013;15(8):e153




Depression is a highly prevalent disorder that often causes substantial functional impairment in daily life, reduction in quality of life, and increased medical service utilization [1-6]. There exist several effective psychological and pharmacological treatments for depression [7]. However, a large proportion of those suffering from this disorder receive inadequate treatment or no treatment at all [8,9]. Cognitive behavioral therapy (CBT) has proven to be as effective as pharmacotherapy in treating mild to moderate depression, with the benefit of reduced rates of relapse [10,11].

Internet-Based Treatment of Depression

The principles and techniques of CBT have been extensively disseminated through self-help books and computer- or Internet-based programs. A substantive body of research shows that Internet-based CBT can be an efficacious treatment of depression (eg, [12-15]). Research also suggests that such interventions are cost-effective compared to face-to-face treatments because they result in symptom reduction and reduced burden of disease for patients and alleviate demands on clinician time and resources [16-18].

Self-help can be self-administered or guided by a therapist, although the active involvement of the therapist in guided self-help is less extensive than in conventional psychological therapy. Studies generally show small to moderate effects of self-administered, unguided CBT in the treatment of depression [19-23], although in some studies unguided interventions have yielded large treatment effects [24,25]. Still, an increasing amount of research has pointed to the importance of support in Internet-based interventions, with interventions offering some degree of support from a professional during treatment generally showing substantially larger treatment effects than interventions involving little or no professional support [26-28]. However, this conclusion is primarily based on meta-analytic results; the results from the few studies directly comparing guided and unguided interventions are mixed [14,24,29]. Overall, guided interventions show moderate to large treatment effects for depression, and the average effect sizes for guided self-help are comparable to the effects of time-limited face-to-face treatment (eg, [13-15,30]). This is further supported by a recent meta-analysis, which found no significant differences in effect between guided self-help and face-to-face therapy [31].

MoodGYM is a free Web-based CBT program developed to prevent and treat mild to moderate depression [32]. Studies have demonstrated the effectiveness of MoodGYM in reducing symptoms of depression and anxiety among public registrants, trial participants, callers to a national helpline service, users of the UK National Health Service portal, adolescent school-based populations, and in Australian and Norwegian student samples [20,24,25,29,33-35]. Positive effects have been shown to be sustained over 12 months [36]. However, few previous trials have investigated the effect of MoodGYM combined with therapist support. A study found that the conjunction of MoodGYM and face-to-face therapy was superior to both MoodGYM alone and for some outcome measures, to time-limited face-to-face therapy alone [29]. Also the results of a cluster-randomized trial suggested positive effects of the combination of MoodGYM and support from general practitioners (GPs) compared to GP care alone [37].

Depression Treatment in Primary Health Care

Most patients with psychological problems will receive most or all of their mental health care in primary care, and findings suggest that many patients prefer to consult their GP for treatment of depression [2,38-40]. Clinical practice guidelines primarily recommend treating mild to moderate depression using psychosocial interventions [41,42], and this is also in accordance with reported patient preferences [43-45]. Nevertheless, structured psychological interventions are infrequently delivered in general practice [46-48] because of time constraints [49-51] and a lack of knowledge and competence among GPs in the delivery of evidence-based psychological interventions [51,52]. The use of CBT-based self-help resources could be a way to improve the delivery of psychological interventions in general practice. This would allow for short consultations and for the clinician to be a facilitator rather than a cognitive therapist. These features could improve feasibility in general practice, where the volume of patients is high and it is essential that interventions are brief and practical.

Aim of the Study

The current study was designed to trial a procedure for depression treatment that could be suitable for implementation in general practice. The project was planned as the first phase of research for this treatment, with the second phase focusing on further evaluation carried out in everyday general practice. The aim was to evaluate the effectiveness and acceptability of a guided self-help intervention combining the MoodGYM program with brief face-to-face therapist support in a sample of primary care patients with mild to moderate symptoms of depression. This was investigated in a randomized controlled trial comparing the guided self-help intervention to a delayed-treatment control condition. The primary hypothesis was that therapist-supported Web-based CBT would lead to a larger reduction in depressive symptoms than the control condition. To determine if the intervention was acceptable to patients, satisfaction with treatment, adherence, and reasons for dropout were investigated.

Study Design

The study was a randomized controlled trial with balanced randomization (1:1). Participants were randomly allocated to a treatment condition comprising 6 weeks of Web-based CBT with therapist support, or to a 6-week waitlist for the same treatment during which time they could also access treatment as usual. The study was conducted at the Department of Psychology at the University of Tromsø where a small self-help outpatient clinic was established. The research protocol was approved by the Regional Committee for Research Ethics in Northern Norway (2011/2163) and the Human Ethics Committee of the Australian National University (ANU). The trial was registered in the Australian New Zealand Clinical Trials Registry (ACTRN12610000257066). The trial is reported in accordance with the CONSORT-EHEALTH [53] (see Multimedia Appendices 1-3).


Participants (N=106) were recruited between October 2010 and October 2012 from GPs, primary care nurses, and from waitlists of primary care referrals at 2 psychiatric outpatient clinics. Calculations of required sample size were based on a power of .80, significance level of .05 (2-sided), and an expected effect size of 0.6 on depressive symptoms at posttest. The estimations necessitated a sample size of 45 participants per group. A median dropout rate between 17% and 19% has been reported for computerized or Web-based treatment programs [54,55]. With a 20% expected dropout, a total sample size of 108 was required to gain sufficient power, yielding group sizes of 54 participants.

Local GPs and primary care nurses were informed about the study both verbally at practice meetings and through written information. They provided patients who they considered as mildly to moderately depressed based on clinical appraisal and/or screening instruments with written information about the project. Potentially eligible patients on waitlists for psychiatric outpatient treatment were identified by clinic staff and subsequently received information by postal mail from the research group. When informing a patient of the project, all recruiters were asked to send a notification to the research group by using a prepaid envelope. The notification simply notified the researchers that a patient had been informed of the project and did not reveal any information about the patient. Patients were provided with general information about the treatment and the aim of the project and detailed information about the methods for handling issues of privacy and anonymity. They were informed that they could expect to commence treatment within 6 weeks of the initial contact. To participate, patients sent in a signed informed consent form providing contact details. Study inclusion criteria were: (1) age 18-65 years, (2) access to the Internet, and (3) a score between 14 and 29 on the Beck Depression Inventory-II (BDI-II), indicating mild to moderate symptoms of depression. During the first months of the study, the protocol was changed by extending the inclusion criterion on the BDI-II to include participants with scores between 10 and 40. This change was because of insufficient recruitment and the clinical appraisal that patients with scores above 30 could possibly benefit from the treatment based on their daily functioning and motivation. In addition, their depression was too mild to assure them other public treatment options. Furthermore, several patients with a BDI-II score below 14 reported a need for treatment. Based on this revised criteria, 7 eligible patients were falsely excluded in the initial phase of the trial. Individuals currently undergoing CBT were excluded, whereas individuals who used antidepressant medication were stabilized for 1 month prior to evaluation of diagnostic eligibility. To maximize the external validity of the trial, a heterogeneous group of patients with depressive symptoms was included, independent of a particular diagnosis. Therefore, medical or psychiatric comorbidities only restricted inclusion when there was a need for immediate treatment of these comorbid conditions (suicidal ideation, current psychosis) or if the conditions were expected to markedly interfere with treatment of the depressive condition (alcohol or drug use disorders).


All outcome measures were based on self-report. Assessments of symptoms of depression, anxiety, and quality of life using paper-and-pencil questionnaires were completed by all participants at baseline, posttreatment, and at 6 months posttreatment (online questionnaires). The control group also completed these inventories before entering online treatment (postwaiting). BDI-II was administered before every consultation during the intervention phase.

The primary outcome measure was the BDI-II, a 21-item measure of severity of depressive symptoms during the past 2 weeks [56]. Each item is rated on a 4-point scale ranging from 0 to 3. Studies consistently support the BDI-II as a reliable, internally consistent, and valid scale for assessing depression both in psychiatric outpatients, the general population, and in primary care settings [56-58]. Several studies have found high correlations between the paper-and-pencil and the computerized/online versions of the BDI-II [59-62]. In the present study, internal consistency (Cronbach alpha) was .78 pretreatment, .93 posttreatment, and .94 at 6-month follow-up, respectively.

The secondary outcome measures consisted of the Beck Anxiety Inventory (BAI), the Hospital Anxiety and Depression Scale (HADS), and 2 measures of quality of life—Satisfaction With Life Scale (SWLS) and the EuroQol Group 5-Dimension Self-Report Questionnaire (EQ-5D)—as well as a measure of treatment satisfaction. The quality of life measures were included during the initial phase of the trial considering that the extension of outcomes beyond symptom measures would strengthen the study.

The BAI is a 21-item measure of anxiety symptom severity [63]. Each item is rated from 0 to 3 depending on symptom severity during the past week. The inventory possesses high internal consistency and reliability, as well as robust convergent and discriminant validity [64-66]. Equivalent psychometric properties have been shown across paper-and-pencil and online formats of the questionnaire, and the 2 formats are highly correlated [59,67]. Cronbach alphas in the present study were .93 at pretreatment, .88 at posttreatment, and .92 at 6-month follow-up, respectively.

The HADS is a 14-item inventory with 2 subscales of 7 items each, measuring depression and anxiety, respectively [68]. Each item is rated on a 0 to 3 scale. The inventory has good to very good construct validity and internal consistency [68-70]. Most factor analyses confirm a 2-factor solution comprising a depression and an anxiety subscale [69]. Paper-and-pencil and Internet administration of the measure yields comparable psychometric properties, but Internet administration may overestimate scores [70,71]. In the present study, Cronbach alpha was .68 and .82 at pretreatment, .82 and .84 at posttreatment, and .87 and .86 at 6-month follow-up for the depression and anxiety subscales, respectively.

The EQ-5D is a generic questionnaire evaluating health-related quality of life [72]. The respondent marks his/her level of functioning (no problems, some problems, extreme problems) for each of 5 dimensions (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression) and rates his/her health state on a visual analog scale (EQ VAS) with the endpoints labeled best imaginable health state and worst imaginable health state, respectively. For the 5 dimensions, a scoring algorithm (the MVH_A1 tariff) based on preference weights was used to aggregate an index score (EQ Index) [73]. The health states “perfect health” (no problems on any dimension) and “dead” are assigned the values of 1 and 0, respectively. The instrument has been demonstrated to discriminate between subgroups of patients with differing severities of mental illness and to capture changes in quality of life associated with improved mental health over time [74]. No significant differences have been found between scores obtained using paper and computerized modes of administration [75,76].

The SWLS measures global life satisfaction as a cognitive-judgemental process, in which individuals assess their quality of life according to their own criteria [77]. The respondent rates on a 7-point Likert scale the degree to which he/she agrees with 5 statements. Several studies confirm the scale’s unidimensional structure and support its sound psychometric properties, including internal consistency, test-retest reliability, as well as convergent and discriminant construct validity [78-80]. Research indicates that Internet data on this measure is as reliable and valid as paper-and-pencil data [81]. Cronbach alpha in this study was .79 at pretreatment, .87 at posttreatment, and .93 at 6-month follow-up.

The Mini-International Neuropsychiatric Interview (MINI) is a short, structured diagnostic interview for identifying the diagnoses of the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition; DSM-IV) and the International Classification of Diseases, Tenth Revision (ICD-10). It consists of 17 modules. A comparison of MINI with other structured clinical interviews shows sensitivities and specificities above 0.70 for most diagnoses [82]. Excellent interrater reliability has been reported. The MINI Interview was used to determine psychiatric comorbidity and for excluding participants with current psychosis or suicidal ideation, as indicated by 17 points or more on the suicidal ideation module.

The Alcohol Use Disorders Identification Test (AUDIT) is a screening instrument consisting of 10 questions about alcohol use in the past 12 months, alcohol dependence symptoms, and alcohol-related problems [83]. Eight items are rated on a 5-point scale (0-4) and 2 items are rated on a 3-point scale (0, 2, 4). A large body of research confirms the favorable internal consistency, reliability, and criterion validity [83-85]. In this study, the scale was used to screen for alcohol use problems. A cutoff score of 20 was chosen to exclude patients in need of further diagnostic evaluation for alcohol dependence [85]. For participants scoring above16, alcohol use was monitored during treatment.

The Drug Use Disorders Identification Test (DUDIT) is an 11-item screening instrument measuring patterns of substance use during the past 12 months, as well as various drug-related problems [86]. Nine items are rated on a 5-point scale (0-4) and 2 items are rated on a 3-point scale (0, 2, 4). In a sample of drug users, the scale has shown good reliability, and it predicts drug dependence with a sensitivity of 0.90 and average specificity of approximately 0.80 [86]. In the present study, the DUDIT was used to screen for drug use disorder. A cutoff score of 25 was used to exclude patients with a high probability of drug dependency.

Satisfaction with treatment was measured by 9 questions that respondents rated on a 5-point scale (1-5, very negative to very positive). The questions concerned their satisfaction with the intervention as a whole and various aspects of the self-help program and follow-up sessions. The general content of the questions was influenced by patient satisfaction questionnaires used in other studies [87-89]. However, the exact content was tailored to tap into aspects of treatment considered important for the purpose of the present investigation. The questions are described in detail in the Results section.

Treatment variables included module completion, number of sessions, treatment duration, session duration (in minutes, not including screening), and total time spent by therapists (time spent outside the consultations was not registered). User data on module completion was registered online and was denoted by a number between 0 and 4, with 0 indicating no use and 4 indicating completion of the module. For the variable time spent by therapists, the amount of missing data was considerable (51.9% for time spent on screening, 14.6% for total time); thus, these data can only be considered estimates. Total time was estimated by imputing mean screening time for missing data concerning screening duration and each individual’s mean session duration for missing data from treatment sessions.


After informed consent was obtained, participants were screened for inclusion through a face-to-face session. A computerized random number generator randomized identification (ID) numbers to the 2 groups (generated by KL). Eligible participants were given ID numbers following a chronological sequence. To ensure equal group sizes, blocked randomization with variable block sizes was used. Patients could not be blinded to group assignment, but were blinded to the status of the waitlist as a control condition.

Screening, enrollment, and treatment were carried out by 2 licensed clinical psychologists (RSH and KL) with basic CBT skills and good knowledge of the MoodGYM program. Both had less than 2 years of experience in clinical practice and no prior experience with Internet-based treatment. The therapists were not blind to the participants’ group allocation. However, steps were taken to blind the evaluation of outcomes by ensuring that posttests were performed by a research assistant unaware of the participants’ allocation assignment.


Participants in both groups were free to access usual primary care treatment, which could include antidepressant medication, informal supportive therapy, or referral to specialist mental health services.

The guided self-help intervention involved 3 components: (1) The Norwegian version of the Web-based CBT program MoodGYM version 3 [90], (2) brief face-to-face therapist support, and (3) tailored emails between sessions. MoodGYM was originally developed at the Australian National University to prevent depression in young people aged between 15 and 25 years. However, data from individuals who used the English version of the program has shown that most users were aged 25 to 44 years, and that the users’ average depression and anxiety scores were elevated compared to the general population [91]. Therefore, the program appears useful for older age groups than originally targeted, and for individuals with elevated levels of depressive and anxious symptomatology. The program consists of 5 modules and a personal workbook containing exercises and assessments. Module 1 through 3 focus on the cognitive model, typical patterns of dysfunctional thinking, and exercises to identify and restructure dysfunctional thinking, as well as behavioral strategies to increase engagement in positive activities. Module 4 focuses on stress and stress reduction and introduces relaxation techniques. Module 5 covers simple problem solving and typical responses to broken relationships. Each module takes approximately 45 to 60 minutes to work through. See Figure 1 for screenshots from the program.

In the first session after screening, participants were introduced to the program, received their trial username and password, and were instructed to work at home with 1 module each week. After each module, participants received face-to-face support (15-30 minutes). The therapists followed a guideline script with 3 compulsory topics for every consultation: (1) monitoring of depression symptoms and discussion of changes, (2) a focus on the important topics and exercises covered by each module and the participants’ experiences of working with it, and (3) introduction of the next module and motivate participants to adhere to the treatment plan. The main focus of the therapist was on reinforcing the efforts made by participants and helping them to relate to the material and to incorporate the use of techniques from the program into their everyday living. If time permitted, participants could also bring up other topics they considered important in relation to their depression. In the concluding session, the experiences and outcomes of treatment were discussed. Therapists aimed to meet participants weekly and to complete the intervention over 7 weeks. However, the interval between sessions and the number of sessions were allowed to vary somewhat to provide flexibility in meeting individual needs. Between sessions, participants received tailored emails aiming to motivate them to work with the self-help program. The emails introduced the next module, and some contained brief advice on how to overcome depressive symptoms (eg, the importance of behavioral activation). Participants did not get a mental health record at the clinic, but a short case summary was sent to their GP (with consent from the participants). Participants considered in need of more extensive treatment throughout or after completing the trial were assisted in the process of referral to specialized mental health services. The Web-based program did not store any personally identifying information about users.

Statistical Analyses

All analyses were carried out using IBM SPSS Statistics version 19 for Windows (IBM Corp, Armonk, NY, USA), except for the power calculation which was performed using G*Power (Heinrich-Heine-Universität, Institut für Experimentelle Psychologie, Düsseldorf, Germany).

Differences between the groups on baseline characteristics were examined by performing chi-square tests for categorical variables and 1-way analysis of variance (ANOVA) for continuous variables. A logistic regression analysis with backward stepwise method was used to explore whether missing data at postintervention (not completing postintervention measures) could be significantly predicted by participant characteristics.

Results on the BDI-II and BAI were analyzed using intention-to-treat (ITT) analyses in which participants are analyzed in the group they were randomized to, irrespective of treatment adherence. Because of missing data at pretest for the remaining secondary measures (3% missing on the HADS, and 17% and 19% missing on SWLS and EQ-5D, respectively), modified ITT analysis was performed including all participants completing the measures at least once. Effects were tested by performing linear mixed-models analysis using the restricted maximum likelihood (REML) estimation procedure and an unstructured covariance matrix. Since linear mixed-models analysis can handle incomplete data, no procedure for imputation of missing data was utilized in the analysis [92]. For the analysis of BDI-II during the treatment phase, random intercepts across participants were estimated, and BDI-IIs from every treatment session were included for the intervention group. Time was coded as 0 for baseline and as number of weeks from baseline for all subsequent measures. To control for differences in treatment duration, the time frame was made comparable between groups by including only measures up to 7 weeks after baseline (the planned time frame for completing the intervention) for the intervention group in the main analysis. For the secondary measures and for the analysis including the 6-month follow-up data on the BDI-II, repeated measures linear mixed-models analysis was performed with occasion (baseline, posttest, 6-month follow-up) as the repeated factor. This procedure was deemed acceptable because linear regression analyses did not find treatment duration to be a significant predictor of symptom change during the treatment phase of the intervention group for any of the secondary measures (beta=-.16 to .28, t29-35=–0.91 to 1.58, P=.12-.92). Scores on the last BDI-II from participants completing 5 or more weeks of treatment but missing formal posttest data (n=8), were included in the analysis, because this was considered to give a more accurate estimate of change over time.

For completers, analyses of covariance (ANCOVA) were performed with postsymptom scores as the dependent variable and preintervention symptom scores and treatment duration as covariates. Effect sizes (Cohen’s d) were calculated for within- and between-group changes based on estimated means or beta coefficients [93]. For the ITT analyses, calculations were based on pooled standard deviations calculated from the square root of each group’s variance parameters from the mixed models analysis: the single variance estimate of the procedure with a random intercept, and the sum of the variance estimates at each time point of interest minus 2 times the covariance estimate between these time points for the repeated procedure [94,95]. For the completer analyses, the square root of the mean square error, equivalent to the pooled standard deviation, was used. A Cohen’s d of 0.2 was considered a small effect, 0.5 a medium effect, and 0.8 or more a large effect [96].

Clinically significant changes on the BDI-II were assessed using the criteria for reliable change and cutoff points developed for the BDI by Seggar et al [97], based on the definition by Jacobson and Truax [98]. Recovery was defined as the combination of reliable improvement (a change of more than 8.5 points on the BDI-II) and endpoint symptom level below the clinical cutoff of 14.3. For those with subclinical symptoms at baseline, reliable improvement required a change of more than 4.6 points and recovery required reliable improvement plus an endpoint less than 4.1 [97]. Change scores between baseline and 7 weeks of treatment were used for the intervention group.

Figure 1. Screenshots from MoodGYM.
View this figure

Sample Characteristics

Figure 2 shows the flow of participants through the trial. Of the 128 individuals screened for participation, 106 (83%) were found eligible. Most participants, including 49 of 52 (94%) in the intervention group and 48 of 54 (89%) in the delayed-treatment group, were recruited from GPs. The remaining participants were recruited from waitlists at 2 outpatient clinics (n=3), from primary care nurses (n=4), and from a low-threshold clinic for youth (n=2). Postintervention measures were completed by 71% (37/52) and 87% (47/54) of the participants in the intervention and the delayed-treatment group, respectively. The 6-month follow-up assessment was completed by 42 participants (81%) in the intervention group and by 34 participants (63%) in the control group.

Group and educational level emerged as significant predictors of dropout at postintervention. The odds of dropping out before the posttest was significantly higher for participants in the intervention group relative to participants in the delayed-treatment group (OR 3.03, 95% CI 1.08-8.47, P=.04), and significantly lower for participants with higher education relative to participants with a lower educational level (OR 0.36, 95% CI 0.13-0.99, P=.048). No other demographic or clinical variables predicted dropout at postintervention.

Table 1 shows the descriptive characteristics of the sample. For most demographic and clinical variables the 2 groups did not differ significantly at baseline (P=.10-.90). However, the groups differed significantly with regard to age (P=.045), with the intervention group being slightly older than the control group. The groups also differed on the variable comorbid anxiety (P=.03), with the number of participants with an anxiety disorder being significantly higher in the control group compared to the intervention group. Baseline scores on all symptom and outcome measures were comparable between the groups (P=.13-.87).

Further exploration of the 2 groups with regard to anxiety level as measured with BAI shows that although a higher proportion of the control group had symptoms corresponding to moderate or severe anxiety compared to the intervention group, the distribution across the categories minimal, mild, moderate, and severe anxiety did not differ significantly (P=.29).

Figure 2. Flow of participants through the trial.
View this figure
Table 1. Participant characteristics at baseline.
Participant characteristicsIntervention
Gender, n (%)

Male15 (28.8)14 (25.9)29 (27.4)

Female37 (71.2)40 (74.1)77 (72.6)
Age (years)

Mean (SD)38.3 (12.2)33.9 (9.9)36.1 (11.3)a

Range19 - 6318 - 5818 - 63
Marital status, n (%)

Married/living together28 (53.8)31 (57.4)59 (55.7)

Separate living2 (3.8)4 (7.4)6 (5.7)

Divorced3 (5.8)5 (9.3)8 (7.5)

Single19 (36.5)14 (25.9)33 (31.1)
Highest educational level,bn (%)

Compulsory school
(9 or 10 years)
6 (11.5)1 (1.9)7 (6.6)

High school16 (30.8)26 (48.1)42 (39.6)

University, 3 years17 (32.7)14 (25.9)31 (29.2)

University, ≥5 years13 (25.0)12 (22.2)25 (23.6)
Employment, n (%)

Employed (full- or part-time)37 (71.2)37 (68.5)74 (69.8)

Student5 (9.6)6 (11.1)11 (10.4)

Long term sick7 (13.5)3 (5.6)10 (9.4)

Homemaker1 (1.9)5 (9.3)6 (5.7)

Unemployed2 (3.8)3 (5.6)5 (4.7)

Sick leave (employed sample)18 (48.6)21 (56.8)39 (52.7)c
Present treatment,dn (%)

Medication13 (25.0)7 (13.0)20 (18.9)

Other treatment7 (13.5)4 (7.4)11 (10.4)

None36 (69.2)44 (81.5)80 (75.5)
Treatment history,en (%)

Earlier32 (61.5)30 (55.6)62 (58.5)

None18 (34.6)23 (42.6)41 (38.7)
Depression current,fn (%)26 (50.0)25 (46.3)51 (48.1)
Number of major depressive episodes,gn (%)

05 (9.6)6 (11.1)11 (10.4)

116 (30.8)18 (33.3)34 (32.1)

2 - 414 (26.9)15 (27.8)29 (27.4)

≥514 (26.9)11 (20.4)25 (23.6)
Comorbidity,hn (%)

Anxiety12 (23.1)23 (42.6)35 (33.0)a

Other axis-I disorder1 (1.9)4 (7.4 %)5 (4.7)
Symptom measures,imean (SD)

AUDIT4.8 (3.8)5.4 (4.3)5.1 (4.0)

DUDIT0.4 (2.2)0.7 (2.5)0.6 (2.3)
Internet Use,jn (%)

Daily50 (96.2)42 (77.8)92 (86.8)

Weekly0 (0.0)3 (5.6)3 (2.8)


b0.9% missing.

c36.8% of the total sample.

dMedication is antidepressants; other treatment is psychological therapy other than CBT.

e2.8% missing.

fMajor depressive episode fulfilling DSM-IV criteria.

g6.6% missing.

hAnxiety includes panic disorder, agoraphobia, social phobia, and generalized anxiety; other axis-1 disorders include bipolar disorder, obsessive-compulsive disorder, bulimia, and posttraumatic stress disorder.

iAUDIT: Alcohol Use Disorders Identification Test, 0.9% missing; DUDIT: Drug Use Disorders Identification Test, 0.9% missing.

j10.4% missing.

Attrition and Adherence

Of the 52 participants in the intervention group, 31 (60%) adhered to the treatment program in that they completed MoodGYM and attended at least 7 sessions. Total nonadherence was 40% (21/52). Reasons for nonadherence are summarized in Figure 2. Overall, the sample starting treatment (n=50) completed a mean of 3.8 (SD 1.7) of the 5 modules, attended a mean of 7.2 (SD 2.3) sessions, with average session duration of 27.7 (SD 6.2) minutes. The average number of weeks in treatment was 11.3 (SD 7.2). Total time spent by therapists ranged from 70 to 506 minutes (mean 242.1, SD 96.6). A previous study found that versions of the MoodGYM program encompassing Module 2 (extended CBT) were associated with greater improvements than versions excluding this module [91]. This suggests that Module 2 may be a particularly important treatment component. Of the 50 participants starting treatment, 86% (n=43) completed 2 or more modules, indicating that they may have completed enough of the treatment program to generate beneficial outcomes.

Depression and Anxiety Symptoms

Table 2 depicts the preintervention, postintervention, and 6-month follow-up means and standard deviations for each group, as well as within-group and between-group effect sizes. The ITT analysis for the primary outcome measure, BDI-II, revealed a significant time by treatment group interaction (F1,244.83=9.55, P=.002, d=0.65). There was also a significant effect of time (F1,245.37=11.87, P=.001). Both groups experienced significant improvements of depressive symptoms during the intervention phase, but this improvement was significantly larger in the intervention group compared to the delayed-treatment group. Because most of the intervention group had not yet completed treatment at 7 weeks, the analysis was repeated using scores from the intervention group up to 8, 10, 12, and 14 weeks. The interaction term remained significant (F1,268.81-333.62=6.34-10.88, P=.01-.001, d=0.54-0.67). Repeated linear mixed-models analysis also found significant different time trends for the groups between posttest and 6-month follow-up (t81.17=2.88, P=.005). During this time, the delayed-treatment group had received treatment, and pairwise comparisons indicated a significant decrease in symptoms in this group (P=.001), whereas level of depressive symptoms in the intervention group remained stable (P=.56).

The ITT analysis for the BAI revealed no significant interaction between treatment group and occasion (F1, 84.31=0.37, P=.69). Pairwise comparisons showed that both groups improved significantly between baseline and posttest (P=.007 and P=.02 for the intervention and control group, respectively; see Table 2). Between posttest and 6-month follow-up, the control group further improved (P=.04), whereas the intervention group remained unchanged (P=.36). Analyses for the HADS subscales yielded significant group by occasion interactions for both the depression subscale (F1,78.05=14.68, P<.001) and the anxiety subscale (F1,78.07=8.10, P=.001). Pairwise comparisons for both subscales found that the intervention group, but not the control group, reduced their scores significantly between pretest and posttest (P<.001 and P=.46, respectively, for the depression subscale and P<.001 and P=.44, respectively, for the anxiety subscale; Table 2). Between posttest and 6-month follow-up, the control group experienced a significant reduction in both depressive and anxiety symptoms (P=.001 and P<.001, respectively), whereas scores did not change significantly for the intervention group for the depressive subscale (P=.07) or for the anxiety subscale (P=.80).

For all reported ITT and modified ITT analyses, baseline anxiety symptoms on the BAI and age were included as covariates, with the exception that baseline BAI scores were not included as a covariate in the analysis of BAI. The effect of BAI scores was consistently significant (P=.02 to P<.001). The effect of age was significant in 2 of 8 models (P=.008-.03).

Completer Analyses

The results of the ANCOVA revealed the same pattern of results. There was a significant effect of group on posttest level of depressive symptoms measured with BDI-II (n=84; P<.001, d=1.09) and with HADS (n=76; P=.002, d=0.95), and level of anxious worry as measured with HADS (n=76; P=.002, d=0.97) after controlling for the effect of baseline symptoms and treatment duration. The results were not significant for anxiety as measured with BAI (n=83; P=.47, d=0.20). Thus, for depression and anxiety measured with HADS, the groups differed significantly in posttest scores after controlling for differences in preintervention scores and treatment duration. Baseline symptoms were significantly related to posttest symptoms for all measures (P<.001), whereas treatment duration did not significantly affect outcome (P=.19-.92).

Quality of Life

For the SWLS, the modified ITT analysis showed a significant interaction between treatment group and occasion (F1,75.75=8.49, P<.001). Pairwise comparisons found a significant increase in satisfaction for the intervention group from pretest to posttest (P<.001), but no such change for the delayed-treatment group (P=.52; see Table 2). Between posttest and 6-month follow-up, there was a significant increase in life satisfaction in the control group (P=.01), whereas the intervention group did not experience significant changes (P=.08). The analyses for EQ Index yielded an overall significant difference in time trends between the groups (F1,67.42=3.55, P=.03). Between pretest and posttest, there was no significant interaction between treatment group and occasion (t68.82=–1.00, P=.32), with both groups improving at comparable rates over time. However, there was a significant group by occasion interaction between posttest and 6-month follow-up (t63.96=–2.66, P=.01, with the control group showing significant improvement (P=.02), whereas there were no significant changes in the intervention group (P=.18). For the EQ VAS, the overall interaction between group and occasion was not significant (F1,71.32=2.25, P=.11). Between pretest and posttest both the intervention group (P<.001) and the control group (P=.03) experienced significantly improved self-reported health state (Table 2), but the interaction between group and occasion was not significant (t66.07=–1.94, P=.06). Between posttest and 6-month follow-up, pairwise comparisons showed a slight, but nonsignificant, improvement in the control group (P=.06), whereas scores in the intervention group remained stable (P=.74).

Completer Analyses

Similar results were found for the completer analyses, in which ANCOVA showed a significant effect of group for the SWLS (n=66) after controlling for baseline levels of life satisfaction and treatment duration (P=.006, d=0.86). There was no significant effect of group on health-related quality of life at posttest, EQ Index (n=63; P=.56, d=0.18), or health state at posttest, EQ VAS (n=61; P=.11, d=0.52), despite a moderate effect size for the latter. The effect of baseline scores was significant for all measures (P≤.002). There were no significant effects of treatment duration (P=.14-.99).

Clinical Significance of Changes in Depressive Symptoms

Table 3 presents data on clinically significant change on the BDI-II, based on scores at 7 weeks for the intervention group. The ITT procedure was used (classifying all who did not start treatment or did not complete their waitlist period as nonresponders). The results of the chi-square tests for the full sample and the sample with BDI scores above clinical cutoff show that significantly more participants recovered in the intervention group compared to the delayed-treatment control group. Conversely, a significantly smaller proportion of the intervention group experienced no change within the intervention period. For the sample fulfilling the criteria for a major depressive episode at baseline, the same trend was evident, but the difference in rate of recovery did not reach significance. The rates of improvement and deterioration were similar in the 2 groups for all analyses. The same analysis carried out after excluding participants (n=8) who during the waitlist or intervention period started or increased their dosage of antidepressant medication, or commenced other psychological treatment, produced similar patterns of results. At 7 weeks of treatment 37 of 52 participants (71%) in the intervention group had completed 2 or more modules, whereas 15% (n=8) had completed treatment.

Table 2. Estimated means (EM),a observed means (OM), observed standard deviations (SD), standard deviations based on linear mixed-models variance estimates (SDm), and effect sizes from pretreatment (pre) to posttreatment (post) and pretreatment to 6-month follow-up (6 m) for the intervention and the delayed-treatment control group.
MeasuresIntervention (n=52)Delayed treatment (n=54)Effect size (Cohen’s d)b




7 weeks15.15c14.20c8.155.8619.0718.638.644.63

6 months13.3912.459.3210.3511.8612.8210.989.56




6 months7.467.076.6110.379.199.419.799.05

HADS Depression1.10–1.17–0.11–0.10–0.53–0.65



6 months5.915.764.054.294.664.853.774.16

HADS Anxiety0.74–0.600.13–0.13–0.52–0.55



6 months6.996.574.163.996.406.944.794.88




6 months18.6619.006.636.4219.7920.006.915.52

EQ-5D VAS0.470.790.42–0.010.710.55



6 months70.1271.7914.6516.0467.7866.4120.8621.56

EQ-5D Index0.220.590.26–0.330.260.74



6 months0.700.720.260.300.780.770.200.21

aEstimated means (except for BAI) are adjusted for the covariates baseline BAI score and age.

bB: between-group effect size; Wi: within-group effect size for the intervention group; Wdt=within-group effect size for the delayed-treatment group.

cEstimated mean after completing treatment=12.43, observed mean after completing treatment=11.34.

Table 3. Proportion of participants reaching the criteria for clinically significant improvement on the Beck Depression Inventory-II (BDI-II) at 7 weeks of treatment and results of chi-square tests (χ2).
Full sample, n (%)
Baseline BDI-II above clinical cutoff,an (%)
Current major depressive episode diagnosis, n (%)

Intervention (n=52)Control (n=54)χ21Intervention (n=42)Control (n=48)χ21Intervention (n=26)Control (n=25)χ21
Recovered17 (32.7)5 (9.3)8.8b15 (35.7)5 (10.4)8.3b8 (30.8)3 (12.0)2.7
Improved8 (15.4)5 (9.3)0.95 (11.9)4 (8.3)0.36 (23.1)3 (12.0)1.1
No change26 (50.0)41 (75.9)7.7b21 (50.0)38 (79.2)8.4b11 (42.3)18 (72.0)4.6c
Deteriorated1 (1.9)3 (5.6)0.31 (2.4)1 (2.1)0.011 (3.8)1 (4.0)0.001




Therapist Effects

Therapist effects were investigated by looking at the interaction between therapist and time for the intervention group. The analyses indicated a significant difference between the 2 therapists when analyzing BDI-II scores up to 7 and 8 weeks of treatment (P=.03-.04). This effect no longer reached significance when including scores up to 10, 12, and 14 weeks of treatment (P=.05-.32). The analyses did not yield differential treatment effects across the 2 therapists for the HADS depression subscale (P=.87), nor for any other outcome measure (P=.50-.94). An exploratory linear regression analysis showed that symptom change in the intervention group was not significantly predicted by the total time spent by the therapists for any outcome measure (beta=–.12 to .26, t29-48=–0.81 to 1.52, P=.14-.96).

Treatment Satisfaction

Table 4 shows the response frequencies for questions regarding satisfaction with the treatment. The results are reported for participants in both groups (intervention group: n=39; delayed-treatment control group: n=26) who completed the full treatment or parts of it. Overall satisfaction with the treatment was high, with 89% (58/65) giving the intervention as a whole a rating of 4 or 5 on a scale with 5 being very satisfied (see Table 4). Most participants also indicated that they would recommend the combined intervention to a friend with a similar problem. The ratings of the MoodGYM program were positive, but somewhat more moderate with between 50% and 60% giving clearly positive ratings (4 or 5 on the 5-point scales, see Table 4) to the benefit of the program, the usefulness of the exercises, and the relevance of the thematic content, and none rating the program as not useful or relevant. The benefit of the treatment sessions and the relationship with the therapist were rated positively by more than 90% (60/65 and 64/65, respectively) of the participants.

Service Use and Work Status After Treatment

Of the 76 participants who completed the follow-up assessment, 45% (19/42) of participants in the intervention group and 38% (13/34) of participants in the control group had received treatment for mental health problems during the 6-month follow-up period. Two participants (3%) had been hospitalized, 19 (25%) had used antidepressant medication (15 currently using), 26 (34%) had received psychological treatment individually or group therapy, and 16 (21%) had received treatment from their GP. Of the 19 participants reporting use of antidepressants, only 6 had commenced this treatment during the follow-up period.

With regard to work status, 6 of 42 respondents (14%) in the intervention group reported that they had been on sick leave at some point in the follow-up period due to feeling tired, stressed, or experiencing mental health problems, whereas 9 of 34 respondents (26%) in the control group reported sick leave during this period.

Table 4. Response frequencies regarding satisfaction and experiences with the treatment (n=65).
ItemSatisfaction/experience scale

Overall satisfaction with the treatment

ScaleVery dissatisfied

Very satisfied

n (%)0 (0.0)0 (0.0)7 (10.8)40 (61.5)18 (27.7)
Change in symptoms

ScaleMuch worse
Neither nor
Much improved

n (%)0 (0.0)2 (3.1)16 (24.6)26 (40.0)21 (32.3)
Would recommend the treatment to a friend with a similar problem

ScaleDefinitely not

Yes, definitely

n (%)0 (0.0)0 (0.0)4 (6.2)21 (32.3)40 (61.5)
Benefit of using MoodGYM

ScaleNo benefit

Highly beneficial

n (%)0 (0.0)4 (6.2)24 (36.9)32 (49.2)5 (7.7)
The usefulness of the exercises in MoodGYM

ScaleNot useful

Very useful

n (%)0 (0.0)5 (7.7)23 (35.4)28 (43.1)9 (13.8)
Relevance of the thematic content of MoodGYMa

ScaleNo relevance

Highly relevant

n (%)0 (0.0)I: 4 (10.3)
C: 1 (3.8)
I: 20 (51.3)
C: 6 (23.1)
I: 10 (25.6)
C: 13 (50.0)
I: 5 (12.8)
C: 6 (23.1)
Benefit of the follow-up sessions

ScaleNo benefit

Highly beneficial

n (%)0 (0.0)0 (0.0)5 (7.7)34 (52.3)26 (40.0)
Satisfaction with the number of sessions

ScaleToo few
Just enough
Too many

n (%)1 (1.5)6 (9.2)55 (84.6)3 (4.6)0 (0.0)
Relationship to the therapist

ScaleVery negative

Very positive

n (%)0 (0.0)0 (0.0)1 (1.5)14 (21.5)50 (76.9)

aP<.05, frequencies reported separately for the intervention (I) and delayed-treatment control (C) groups.

Principal Findings

The results of the present study indicate that a guided self-help intervention combining the MoodGYM program with face-to-face therapist support can be effective in reducing depressive symptoms for a sample of mildly to moderately depressed individuals recruited from primary care. The intervention also had significant positive effects on symptoms of anxious worry, and participants experienced significant improvements in global satisfaction with life. At 6-month follow-up, positive gains in terms of reduction of depressive and anxious symptoms were largely maintained, whereas improvements in life satisfaction were partly maintained. The rate of nonadherence (40%) was moderate and the evaluations of the treatment as a whole were predominately positive.

These findings are consistent with previous research in which favorable outcomes have been shown for treatments combining MoodGYM and face-to-face support from a professional [29,37]. The trials are not fully comparable, though, because the present study used a delayed-treatment control condition, whereas both previous trials used comparison groups that received more active treatments. This makes direct comparisons of between-group effect sizes difficult. However, the effect of guided self-help for mild to moderate depression using other Internet-based programs has generally been in the moderate to large range [19,27]. The magnitude of the between-group effect size on depression measured with BDI-II in this study was within this range, but somewhat below average, whereas the effects on SWLS and on depression and anxiety measured with HADS were commensurable to effects found previously for similar measures [12-14,27,99,100]. Also, the rates of recovery and of improvement and recovery combined (33% and 48%, respectively) are in good accordance with results of prior investigations, in which clinically significant improvement and recovery have varied between 25% and 50% [12,14,15,30,100]. The nature of the guidance provided in these studies is somewhat heterogeneous, with some studies, such as the present study, defining guidance as active engagement in the therapeutic process, whereas several other studies have focused primarily on providing feedback and encouragement. However, there is no indication of significantly differential treatment effects depending on the nature of the guidance to date [19,100]. The somewhat smaller between-group differences of the present study must be seen in relation to the relatively high degree of positive change in the control group, with effect sizes being in the small to moderate range for several measures (see Table 2). Almost half of the sample did not fulfill the criteria for a major depressive episode on enrollment in the trial, and previous studies of minor depression have shown high rates of placebo response in primary care patients [101] and substantial likelihood of spontaneous remission in the general population [102]. In the present study, the control group was also free to access usual treatment in general practice during the waiting period. In addition, prior to entering the study control participants participated in a screening session in which they had the opportunity to describe their problems, something that could have a therapeutic effect per se. These factors may partly explain the positive gains in the delayed-treatment group and, hence, the modest differences in outcome between the groups.

The results of the 6-month follow-up are also encouraging in that improvements of depression and anxiety symptoms were largely maintained, and the number of participants reporting sick leave due to mental health problems was substantially smaller during follow-up compared to baseline (20% and 53%, respectively). The proportion of participants using antidepressant medication at baseline and during the follow-up period was comparable, but there was a considerable increase in the number of participants who had accessed psychological treatment during follow-up (34%) compared to baseline (10%). This is, however, not surprising because many participants had already been referred to such treatment when entering the project, but may still have contributed to further improvements and maintenance of symptom reduction. For some measures, particularly the depression subscale of the HADS and the SWLS, there was a tendency toward an increase in symptoms and lowering of life satisfaction during the follow-up period. The inclusion of booster sessions after the completion of the active treatment phase could be a measure to accomplish continued use of helpful techniques and skills and the prevention of symptom relapse. Further research is needed to clarify this issue.

The results of the present trial are also consistent with research suggesting that MoodGYM, despite its main focus on depressive thought content, can have significant positive effects on anxiety symptoms [29,35,103]. In the present study, significant treatment effects were found for anxiety symptoms measured with HADS, but not with BAI. The questions on the HADS focus primarily on symptoms such as worry, nervousness, and not being able to relax. This is in good accordance with the core symptoms of the Goldberg Anxiety Scale [104], which has been used in several studies with MoodGYM. In comparison, the BAI incorporates both these subjective anxiety symptoms, as well as 3 more somatic symptom clusters: neurophysiological, autonomic, and panic [63]. The MoodGYM program focuses primarily on restructuring dysfunctional thinking and does not include an introduction to the physiology of anxiety or other treatment techniques for anxiety. Thus, the present results, with the program showing effects on anxious worry but not on more physiological symptoms, seem to be in-line with the thematic focus of the program.

The adherence rate in the present study is comparable to that observed in other online guided interventions, in which adherence varied between 55% and 75% [13,14,23,30,99]. The rates of adherence are also comparable to those seen in other psychotherapy research and in regular clinical practice [105-107]. This level of adherence in the current study, and the high proportion (89%) of completers reporting being satisfied with the treatment, points to the acceptability of the intervention. The evaluation of the MoodGYM program was somewhat more moderate with between 50% and 60% allocating an unambiguous positive rating to the benefit and relevance of the program. This moderate level of satisfaction in the present adult sample may arise from the fact that the MoodGYM program was originally targeted at youth and young adults. Although the therapists emphasized the applicability of the principles for all age groups when introducing the intervention, and many participants managed well to make use of the content, participants frequently characterized the program as “too young.” As these aspects of treatment were not formally measured, these factors require further investigation to be properly elucidated.

Strengths and Limitations

The current study was designed to trial a treatment procedure prior to its evaluation in general practice. Therefore, we sought to ensure a high level of internal validity while at the same time aiming to increase external validity by reflecting the heterogeneity of patients in real clinical practice. One strength of the study is the relatively heterogeneous sample of participants with regard to the range of depression and anxiety symptoms. There was also substantial comorbidity with anxiety disorders, although lower than rates found in population-based studies [108,109]. The fact that 83% of screened participants were found eligible also indicates that the sample is representative of those who opted for this choice of treatment.

An overarching focus in designing the intervention was feasibility for implementation in general practice. Studies have suggested that GPs may find the implementation of CBT techniques too time-consuming [49,110]. Therefore, sessions were primarily supportive and structured by the Web-based program. To allow for flexibility and increase feasibility, a guideline script rather than a more comprehensive manual guided each consultation. This lack of rigid standardization may have introduced some variability in treatment fidelity.

Blinding of participants was not possible for obvious reasons in this trial. However, the control group was blinded to the status of the waitlist as a control condition. Waitlists for treatment is the norm in Norwegian mental health care, and the short wait for the present treatment compared to other treatment options, may have minimized negative effects (“nocebo effects”) in the control group.

The present study also has several limitations that need to be addressed. First, the design of the study with only 1 intervention group receiving a compound of intervention elements does not allow for tests of the specific contribution of MoodGYM and the face-to-face consultations. Second, the lack of allocation concealment and blinding, and the role of the first and second authors as therapists in the trial introduced a risk of bias that may have inflated the treatment effects. Unfortunately, resource constraints prevented the use of independent therapists. The use of self-report measures rather than therapist assessments does alleviate this problem to some degree. Biased outcome assessments were further prevented by ensuring that a research assistant without knowledge of the participants’ condition assignment collected posttest data. Third, the sole reliance on self-report is a limitation in itself. Independent preassessments and postassessments by a clinician blinded to condition allocation would have been preferable and would have strengthened the results. Fourth, at preintervention there was a lack of comparability in diagnosed anxiety between the groups, with a significantly larger proportion of the control group fulfilling the criteria for an anxiety disorder. Despite this difference in diagnosed anxiety, scores on 2 different anxiety scales (HADS and BAI) were not significantly different, which suggests comparable anxiety levels in the groups. To minimize the effect of differences in anxiety, all primary analyses were controlled for anxiety level as measured with BAI, for which the observed difference was most significant. Fifth, the use of an unequal number of assessments of depressive symptoms in the 2 groups, with the intervention group having weekly assessments and the control group only completing a pretest and posttest, may have resulted in more favorable effects in the intervention group because of measurement effects. Previous studies of nonclinical samples have indicated that scores on the BDI tend to decrease with repeated administration [111,112]. Whether this holds for clinical samples is less certain. In this study, the effect of repeated measurement cannot be clearly distinguished from the treatment effect. However, comparable effects were also found for symptoms assessed only preintervention and postintervention in both groups. This indicates that the treatment had beneficial effects over and above possible measurement effects. Still, in light of this limitation, the results must be interpreted with caution. Sixth, the use of different administration formats for the assessments of the treatment phase and follow-up, (paper-and-pencil vs online questionnaires, respectively) can potentially introduce measurement bias. Although the 2 formats correlate highly, a previous study reported a significant difference in mean scores on the BDI-II and BAI, which makes switching of formats problematic [59]. Despite this limitation, the results should not be considered weakened for most measures because the direction of differences has generally suggested that online versions tend to inflate estimations of symptom severity and lower ratings of quality of life [59,71,76], with the exception of BAI, for which Carlbring et al [59] found that means on the online version were lower compared to the paper-and-pencil version. The reliability of the 6-month follow-up results for the BAI may, therefore, be limited. Seventh, the multiplicity of outcomes increases the risk of type I errors. However, the main findings of the present trial would still be significant when employing the Bonferroni correction. This indicates the robustness of the findings. Finally, although the heterogeneity of the sample and the recruitment from primary care is a strength, the generalizability of the results is uncertain because the sample was a self-selected group. Based on the notifications by the GPs when informing a patient of the study, the estimated uptake (meeting up for screening) was 39% (128 of 325 who received information), which is slightly greater than the median uptake for computerized CBT [55]. Considering the extra barriers imposed by the research activities, this uptake rate is relatively high and indicates the possible acceptability of this treatment among depressed primary care patients. It also strengthens the generalizability of the results, by indicating that the self-selected group may be representative of a considerable proportion of the targeted group of primary care patients.

Potential Clinical Implications and Further Research

The positive treatment effects found for the intervention in the present study are encouraging and suggest that this intervention may have a potential for use in a stepped-care approach. The demand for mental health treatment is higher than what can be met by the current number of trained clinicians [113]. To increase availability of treatment, beneficial interventions must be delivered as efficiently as possible to as many people as possible. The present intervention is time-limited, and because the CBT elements are largely delivered by the program, primary care therapists with some training in CBT and MoodGYM, should be able to provide adequate guidance. In fact, studies show that guidance may be delivered effectively not only by trained clinicians, but also by mental health workers with limited experience and by computer technicians [114,115]. Thus, dissemination of the current intervention to regular primary care could be a step toward increasing access to psychological therapies. However, the moderate ratings of the benefit and relevance of the content of the Web-based program by an adult population points to the need for a variety of Web-based tools to make such treatments acceptable for a wider audience.

For practical reasons, we chose to use psychologists for this first evaluation. Therefore, further research is needed to determine if the present intervention would be as effective and acceptable in regular clinical practice when delivered by GPs or other primary care therapists. It may also be noted that the present intervention was more time-intensive than most other guided self-help interventions. However, since the role of the clinician was mainly supportive and facilitative and the main therapeutic input was delivered through a standardized treatment package, the intervention was regarded conceptually as a guided self-help intervention [31]. Similar effects have been found for low- and high-intensity guided Internet-based psychotherapy [19]. Further research should investigate if the present intervention with more limited therapist support could yield similar effects.


Despite its limitations, the present study indicates that an intervention combining the MoodGYM program with therapist support can be an effective treatment of depression in a sample of primary care patients. The intervention not only alleviates depressive symptoms, but also has positive and significant effects on symptoms of anxious worry and global satisfaction with life. Positive gains in terms of reduction of depressive and anxious symptoms were largely maintained at 6-month follow-up, and improvements in life satisfaction were partly maintained. Moderate rates of nonadherence and predominately positive evaluations of the treatment as a whole also indicates the acceptability of the intervention. The intervention was designed to be suitable for implementation in primary health care, and could have a potential for use in a stepped-care approach. However, further research is necessary to determine whether it is equally effective when delivered in regular primary health care and whether the inclusion of booster sessions could further improve symptom maintenance. Further research is also needed to investigate whether the intervention is truly acceptable for the wider group of primary care patients and whether it is considered feasible and acceptable by GPs or other primary care therapists.


This study was funded by the Research Council of Norway (196423/V50). MoodGYM was provided by the Australian National University. The authors would like to acknowledge the contribution of Anthony Bennett, IT manager of the e-hub, the e-mental health research and development group at the Centre for Mental Health Research at ANU; Kylie Bennett, Development Manager at e-hub, who provided technical expertise in administration of the trial; and Ada Tam, e-hub Project Officer at ANU, who provided technical support for users in the trial. We would also like to thank Jan Abel Olsen at the Institute of Community Medicine at the University of Tromsø for contributing with his expertise on the EQ-5D scale. Last, but not least, we would like to give a special thanks to those participating in the trial.

Conflicts of Interest

KG is one of the authors of the MoodGYM intervention evaluated in the trial, and works for The Australian National University who provides free access to the MoodGYM program. KW and ME contributed in the process of translating MoodGYM into Norwegian. RSH, KL, TW, and NK have no financial or nonfinancial interests to declare in relation to this study.

Multimedia Appendix 1

CONSORT-EHEALTH checklist V1.6.2 [53].

PDF File (Adobe PDF File), 999KB

Multimedia Appendix 2

Trial Protocol as requested in CONSORT-EHEALTH [53].

PDF File (Adobe PDF File), 232KB

Multimedia Appendix 3

Trial information to participants as requested in CONSORT-EHEALTH [53].

PDF File (Adobe PDF File), 115KB

  1. Ayuso-Mateos JL, Vázquez-Barquero JL, Dowrick C, Lehtinen V, Dalgard OS, Casey P, ODIN Group. Depressive disorders in Europe: prevalence figures from the ODIN study. Br J Psychiatry 2001 Oct;179:308-316 [FREE Full text] [Medline]
  2. Wittchen HU, Jacobi F. Size and burden of mental disorders in Europe--a critical review and appraisal of 27 studies. Eur Neuropsychopharmacol 2005 Aug;15(4):357-376. [CrossRef] [Medline]
  3. Kringlen E, Torgersen S, Cramer V. A Norwegian psychiatric epidemiological study. Am J Psychiatry 2001 Jul;158(7):1091-1098. [Medline]
  4. Kessler RC, Angermeyer M, Anthony JC, DE Graaf R, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry 2007 Oct;6(3):168-176 [FREE Full text] [Medline]
  5. Kilbourne AM, Daugherty B, Pincus HA. What do general medical guidelines say about depression care? Depression treatment recommendations in general medical practice guidelines. Curr Opin Psychiatry 2007 Nov;20(6):626-631. [CrossRef] [Medline]
  6. Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jönsson B, et al. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol 2011 Sep;21(9):655-679. [CrossRef] [Medline]
  7. Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol 2008 Dec;76(6):909-922. [CrossRef] [Medline]
  8. Bebbington P, Brugha T, Meltzer H, Jenkins R, Ceresa C, Farrell M, et al. Neurotic disorders and the receipt of psychiatric treatment. Int Rev Psychiatry 2003;15(1-2):108-114. [CrossRef] [Medline]
  9. Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, et al. The prevalence and correlates of untreated serious mental illness. Health Serv Res 2001 Dec;36(6 Pt 1):987-1007 [FREE Full text] [Medline]
  10. Churchill R, Hunot V, Corney R, Knapp M, McGuire H, Tylee A, et al. A systematic review of controlled trials of the effectiveness and cost-effectiveness of brief psychological treatments for depression. Health Technol Assess 2001;5(35):1-173 [FREE Full text] [Medline]
  11. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev 2006 Jan;26(1):17-31. [CrossRef] [Medline]
  12. Ruwaard J, Schrieken B, Schrijver M, Broeksteeg J, Dekker J, Vermeulen H, et al. Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up. Cogn Behav Ther 2009;38(4):206-221. [CrossRef] [Medline]
  13. Andersson G, Bergström J, Holländare F, Carlbring P, Kaldo V, Ekselius L. Internet-based self-help for depression: randomised controlled trial. Br J Psychiatry 2005 Nov;187:456-461 [FREE Full text] [CrossRef] [Medline]
  14. Berger T, Hämmerli K, Gubser N, Andersson G, Caspar F. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther 2011;40(4):251-266. [CrossRef] [Medline]
  15. Johansson R, Sjöberg E, Sjögren M, Johnsson E, Carlbring P, Andersson T, et al. Tailored vs. standardized Internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS One 2012;7(5):e36905 [FREE Full text] [CrossRef] [Medline]
  16. McCrone P, Knapp M, Proudfoot J, Ryden C, Cavanagh K, Shapiro DA, et al. Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry 2004 Jul;185:55-62 [FREE Full text] [Medline]
  17. Marks IM, Mataix-Cols D, Kenwright M, Cameron R, Hirsch S, Gega L. Pragmatic evaluation of computer-aided self-help for anxiety and depression. Br J Psychiatry 2003 Jul;183:57-65 [FREE Full text] [Medline]
  18. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan;34(1):9-17. [Medline]
  19. Titov N. Internet-delivered psychotherapy for depression in adults. Curr Opin Psychiatry 2011 Jan;24(1):18-23. [CrossRef] [Medline]
  20. Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res 2004 Dec 22;6(4):e46 [FREE Full text] [CrossRef] [Medline]
  21. Cuijpers P, Donker T, Johansson R, Mohr DC, van Straten A, Andersson G. Self-guided psychological treatment for depressive symptoms: a meta-analysis. PLoS One 2011;6(6):e21274 [FREE Full text] [CrossRef] [Medline]
  22. Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res 2009;11(2):e15 [FREE Full text] [CrossRef] [Medline]
  23. Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009;38(4):222-234 [FREE Full text] [CrossRef] [Medline]
  24. Farrer L, Christensen H, Griffiths KM, Mackinnon A. Internet-based CBT for depression with and without telephone tracking in a national helpline: randomised controlled trial. PLoS One 2011;6(11):e28099 [FREE Full text] [CrossRef] [Medline]
  25. Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacy of unguided Internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin Psychol Psychother 2013;20(1):10-27. [CrossRef] [Medline]
  26. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
  27. Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother 2012 Jul;12(7):861-9; quiz 870. [CrossRef] [Medline]
  28. Palmqvist B, Carlbring P, Andersson G. Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? Expert Rev Pharmacoecon Outcomes Res 2007 Jun;7(3):291-297. [CrossRef] [Medline]
  29. Sethi S, Campbell AJ, Ellis LA. The use of computerized self-help packages to treat adolescent depression and anxiety. Journal of Technology in Human Services 2010 Aug 31;28(3):144-160. [CrossRef]
  30. Perini S, Titov N, Andrews G. Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial. Aust N Z J Psychiatry 2009 Jun;43(6):571-578. [CrossRef] [Medline]
  31. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010 Dec;40(12):1943-1957. [CrossRef] [Medline]
  32. Groves C, Griffiths K, Christensen H. Working out MoodGYM: A User’s Guide. Canberra, Australia: Centre for Mental Health Research; 2003.
  33. Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res 2013;15(1):e2 [FREE Full text] [CrossRef] [Medline]
  34. Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the Internet: randomised controlled trial. BMJ 2004 Jan 31;328(7434):265 [FREE Full text] [CrossRef] [Medline]
  35. Calear AL, Christensen H, Mackinnon A, Griffiths KM, O'Kearney R. The YouthMood Project: a cluster randomized controlled trial of an online cognitive behavioral program with adolescents. J Consult Clin Psychol 2009 Dec;77(6):1021-1032. [CrossRef] [Medline]
  36. Mackinnon A, Griffiths KM, Christensen H. Comparative randomised trial of online cognitive-behavioural therapy and an information website for depression: 12-month outcomes. Br J Psychiatry 2008 Feb;192(2):130-134 [FREE Full text] [CrossRef] [Medline]
  37. Hickie IB, Davenport TA, Luscombe GM, Moore M, Griffiths KM, Christensen H. Practitioner-supported delivery of Internet-based cognitive behaviour therapy: evaluation of the feasibility of conducting a cluster randomised trial. Med J Aust 2010 Jun 7;192(11 Suppl):S31-S35. [Medline]
  38. Wang PS, Berglund P, Kessler RC. Recent care of common mental disorders in the United States : prevalence and conformance with evidence-based recommendations. J Gen Intern Med 2000 May;15(5):284-292 [FREE Full text] [Medline]
  39. Jorm AF. Mental health literacy. Public knowledge and beliefs about mental disorders. Br J Psychiatry 2000 Nov;177:396-401 [FREE Full text] [Medline]
  40. Bebbington P, Meltzer H, Brugha T, Farrell M, Jenkins R, Ceresa C, et al. Unequal access and unmet need: neurotic disorders and the use of primary care services. Int Rev Psychiatry 2003;15(1-2):115-122. [CrossRef] [Medline]
  41. Helsedirektoratet. Nasjonale retningslinjer for diagnostisering og behandling av voksne med depresjon i primær- og spesialisthelsetjenesten. 2009.   URL: http:/​/www.​​publikasjoner/​nasjonale-retningslinjer-for-diagnostisering-og-behandling-av-voksne-med-depresjon-i-primer--og-spesialisthelsetjenesten/​Sider/​default.​aspx [accessed 2013-05-08] [WebCite Cache]
  42. National Institute for Health and Clinical Excellence. Depression in adults: the treatment and management of depression in adults. 2009.   URL: [accessed 2013-05-08] [WebCite Cache]
  43. Angermeyer MC, Matschinger H. Public attitude towards psychiatric treatment. Acta Psychiatr Scand 1996 Nov;94(5):326-336. [Medline]
  44. Priest RG, Vize C, Roberts A, Roberts M, Tylee A. Lay people's attitudes to treatment of depression: results of opinion poll for Defeat Depression Campaign just before its launch. BMJ 1996 Oct 5;313(7061):858-859 [FREE Full text] [Medline]
  45. van Schaik DJ, Klijn AF, van Hout HP, van Marwijk HW, Beekman AT, de Haan M, et al. Patients' preferences in the treatment of depressive disorder in primary care. Gen Hosp Psychiatry 2004;26(3):184-189. [CrossRef] [Medline]
  46. King M, Davidson O, Taylor F, Haines A, Sharp D, Turner R. Effectiveness of teaching general practitioners skills in brief cognitive behaviour therapy to treat patients with depression: randomised controlled trial. BMJ 2002 Apr 20;324(7343):947-950 [FREE Full text] [Medline]
  47. Davidsen A. Experiences of carrying out talking therapy in general practice: a qualitative interview study. Patient Educ Couns 2008 Aug;72(2):268-275. [CrossRef] [Medline]
  48. Backenstrass M, Joest K, Rosemann T, Szecsenyi J. The care of patients with subthreshold depression in primary care: is it all that bad? A qualitative study on the views of general practitioners and patients. BMC Health Serv Res 2007;7:190 [FREE Full text] [CrossRef] [Medline]
  49. Wiebe E, Greiver M. Using cognitive behavioural therapy in practice: qualitative study of family physicians' experiences. Can Fam Physician 2005 Jul;51:992-993 [FREE Full text] [Medline]
  50. Telford R, Hutchinson A, Jones R, Rix S, Howe A. Obstacles to effective treatment of depression: a general practice perspective. Fam Pract 2002 Feb;19(1):45-52 [FREE Full text] [Medline]
  51. Mykletun A, Knudsen AK, Tangen T, Overland S. General practitioners' opinions on how to improve treatment of mental disorders in primary health care. Interviews with one hundred Norwegian general practitioners. BMC Health Serv Res 2010;10:35 [FREE Full text] [CrossRef] [Medline]
  52. Blashki G, Parsons J, Morgan H, Hickie IB, Davenport TA. Providing psychological treatments in general practice. Rationale and practicalities. Aust Fam Physician 2003 Aug;32(8):625-630. [Medline]
  53. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
  54. Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disorders. Br J Clin Psychol 2010 Nov;49(Pt 4):455-471. [CrossRef] [Medline]
  55. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [CrossRef] [Medline]
  56. Beck AT, Steer RA, Brown GK. BDI-II, Beck Depression Inventory: Manual. San Antonio, TX: Psychological Corp; 1996.
  57. Arnau RC, Meagher MW, Norris MP, Bramson R. Psychometric evaluation of the Beck Depression Inventory-II with primary care medical patients. Health Psychol 2001 Mar;20(2):112-119. [Medline]
  58. Dozois DJA, Dobson KS, Ahnberg JL. A psychometric evaluation of the Beck Depression Inventory-II. Psychological Assessment 1998;10(2):83-89. [CrossRef]
  59. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, Öst LG, et al. Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Computers in Human Behavior 2007 May;23(3):1421-1434. [CrossRef]
  60. Holländare F, Andersson G, Engström I. A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. J Med Internet Res 2010;12(5):e49 [FREE Full text] [CrossRef] [Medline]
  61. Naus MJ, Philipp LM, Samsi M. From paper to pixels: A comparison of paper and computer formats in psychological assessment. Computers in Human Behavior 2009 Jan;25(1):1-7. [CrossRef]
  62. Schulenberg SE, Yutrzenka BA. Equivalence of computerized and conventional versions of the Beck Depression Inventory-II (BDI-II). Curr Psychol 2001 Sep;20(3):216-230. [CrossRef]
  63. Beck AT, Steer RA. Beck Anxiety Inventory: Manual. San Antonio, TX: The Psychological Corporation; 1993.
  64. Fydrich T, Dowdall D, Chambless DL. Reliability and validity of the Beck Anxiety Inventory. Journal of Anxiety Disorders 1992 Jan;6(1):55-61. [CrossRef]
  65. Steer RA, Ranieri WF, Beck AT, Clark DA. Further evidence for the validity of the Beck Anxiety Inventory with psychiatric outpatients. Journal of Anxiety Disorders 1993 Jul;7(3):195-205. [CrossRef]
  66. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988 Dec;56(6):893-897. [Medline]
  67. Hedman E, Ljótsson B, Rück C, Furmark T, Carlbring P, Lindefors N, et al. Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation. Computers in Human Behavior 2010 Jul;26(4):736-740. [CrossRef]
  68. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983 Jun;67(6):361-370. [Medline]
  69. Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res 2002 Feb;52(2):69-77. [Medline]
  70. Andersson G, Kaldo-Sandström V, Ström L, Strömgren T. Internet administration of the Hospital Anxiety and Depression Scale in a sample of tinnitus patients. J Psychosom Res 2003 Sep;55(3):259-262. [Medline]
  71. McCue P, Buchanan T, Martin CR. Screening for psychological distress using Internet administration of the Hospital Anxiety and Depression Scale (HADS) in individuals with chronic fatigue syndrome. Br J Clin Psychol 2006 Nov;45(Pt 4):483-498. [CrossRef] [Medline]
  72. EuroQol Group. About EQ-5D. 2013.   URL: [accessed 2013-05-08] [WebCite Cache]
  73. Dolan P. Modeling valuations for EuroQol health states. Med Care 1997 Nov;35(11):1095-1108. [Medline]
  74. Lamers LM, Bouwmans CA, van Straten A, Donker MC, Hakkaart L. Comparison of EQ-5D and SF-6D utilities in mental health patients. Health Econ 2006 Nov;15(11):1229-1236. [CrossRef] [Medline]
  75. Shervin N, Dorrwachter J, Bragdon CR, Shervin D, Zurakowski D, Malchau H. Comparison of paper and computer-based questionnaire modes for measuring health outcomes in patients undergoing total hip arthroplasty. J Bone Joint Surg Am 2011 Feb 2;93(3):285-293. [CrossRef] [Medline]
  76. Bushnell DM, Reilly MC, Galani C, Martin ML, Ricci JF, Patrick DL, et al. Validation of electronic data capture of the Irritable Bowel Syndrome--Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol. Value Health 2006;9(2):98-105. [CrossRef] [Medline]
  77. Diener E, Emmons RA, Larsen RJ, Griffin S. The Satisfaction With Life Scale. J Pers Assess 1985 Feb;49(1):71-75. [CrossRef] [Medline]
  78. Pavot W, Diener E. Review of the Satisfaction With Life Scale. Psychological Assessment 1993;5(2):164-172. [CrossRef]
  79. Clench-Aas J, Nes RB, Dalgard OS, Aarø LE. Dimensionality and measurement invariance in the Satisfaction with Life Scale in Norway. Qual Life Res 2011 Oct;20(8):1307-1317 [FREE Full text] [CrossRef] [Medline]
  80. Glaesmer H, Grande G, Braehler E, Roth M. The German Version of the Satisfaction With Life Scale (SWLS): psychometric properties, validity, and population-based norms. European Journal of Psychological Assessment 2011 May 3;27(2):127-132. [CrossRef]
  81. Meyerson P, Tryon WW. Validating Internet research: a test of the psychometric equivalence of Internet and in-person samples. Behav Res Methods Instrum Comput 2003 Nov;35(4):614-620. [Medline]
  82. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59 Suppl 20:22-33;quiz 34. [Medline]
  83. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II. Addiction 1993 Jun;88(6):791-804. [Medline]
  84. Reinert DF, Allen JP. The alcohol use disorders identification test: an update of research findings. Alcohol Clin Exp Res 2007 Feb;31(2):185-199. [CrossRef] [Medline]
  85. Babor TF, Higgins-Biddle JC, Saunders JB, Monteiro MG. The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Care 2nd edition. 2001.   URL: [accessed 2013-05-10] [WebCite Cache]
  86. Berman AH, Bergman H, Palmstierna T, Schlyter F. Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample. Eur Addict Res 2005;11(1):22-31. [CrossRef] [Medline]
  87. Cavanagh K, Shapiro DA, Van Den Berg S, Swain S, Barkham M, Proudfoot J. The acceptability of computer-aided cognitive behavioural therapy: a pragmatic study. Cogn Behav Ther 2009;38(4):235-246. [CrossRef] [Medline]
  88. Garratt A, Bjørngaard JH, Dahle KA, Bjertnaes OA, Saunes IS, Ruud T. The Psychiatric Out-Patient Experiences Questionnaire (POPEQ): data quality, reliability and validity in patients attending 90 Norwegian clinics. Nord J Psychiatry 2006;60(2):89-96. [CrossRef] [Medline]
  89. Nasjonalt kunnskapssenter for helsetjenesten. PasOpp: hva er erfaringene dine som bruker av poliklinikk i psykiatrisk helsevern?. 2007.   URL: http:/​/www.​​Sp%C3%B8rreskjemabanken/​Spesialisthelsetjeneste+-+Psykisk+helsevern/​Voksne%2C+psykisk+helsevern%2C+poliklinikk.​9716.​cms [accessed 2013-05-10] [WebCite Cache]
  90. Australian National University, Centre for Mental Health Research. The MoodGYM training program: mark III. 2008.   URL: [accessed 2013-05-10] [WebCite Cache]
  91. Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol Med 2006 Dec;36(12):1737-1746. [CrossRef] [Medline]
  92. Twisk JWR. Applied multilevel analysis: a practical guide. Cambridge, UK: Cambridge University Press; 2006.
  93. Silfvernagel K, Carlbring P, Kabo J, Edström S, Eriksson J, Månson L, et al. Individually tailored Internet-based treatment for young adults and adults with panic attacks: randomized controlled trial. J Med Internet Res 2012;14(3):e65 [FREE Full text] [CrossRef] [Medline]
  94. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods 2002 Mar;7(1):105-125. [Medline]
  95. Thorsell J, Finnes A, Dahl J, Lundgren T, Gybrant M, Gordh T, et al. A comparative study of 2 manual-based self-help interventions, acceptance and commitment therapy and applied relaxation, for persons with chronic pain. Clin J Pain 2011 Oct;27(8):716-723. [CrossRef] [Medline]
  96. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: L Erlbaum Associates; 1988.
  97. Seggar LB, Lambert MJ, Hansen NB. Assessing clinical significance: Application to the Beck Depression Inventory. Behavior Therapy 2002 Mar;33(2):253-269. [CrossRef]
  98. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline]
  99. Choi I, Zou J, Titov N, Dear BF, Li S, Johnston L, et al. Culturally attuned Internet treatment for depression amongst Chinese Australians: a randomised controlled trial. J Affect Disord 2012 Feb;136(3):459-468. [CrossRef] [Medline]
  100. Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, Oberg J, et al. Internet administered guided self-help versus individualized e-mail therapy: A randomized trial of two versions of CBT for major depression. Behav Res Ther 2010 May;48(5):368-376. [CrossRef] [Medline]
  101. Barrett JE, Williams JW, Oxman TE, Frank E, Katon W, Sullivan M, et al. Treatment of dysthymia and minor depression in primary care: a randomized trial in patients aged 18 to 59 years. J Fam Pract 2001 May;50(5):405-412. [Medline]
  102. Hermens ML, van Hout HP, Terluin B, van der Windt DA, Beekman AT, van Dyck R, et al. The prognosis of minor depression in the general population: a systematic review. Gen Hosp Psychiatry 2004;26(6):453-462. [CrossRef] [Medline]
  103. Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. J Med Internet Res 2002;4(1):e3 [FREE Full text] [CrossRef] [Medline]
  104. Goldberg D, Bridges K, Duncan-Jones P, Grayson D. Detecting anxiety and depression in general medical settings. BMJ 1988 Oct 8;297(6653):897-899 [FREE Full text] [Medline]
  105. Bados A, Balaguer G, Saldaña C. The efficacy of cognitive-behavioral therapy and the problem of drop-out. J Clin Psychol 2007 Jun;63(6):585-592. [CrossRef] [Medline]
  106. Watkins E, Williams R. The efficacy of cognitive behavioral therapy. In: Checkley S, editor. The Management of Depression. Oxford, UK: Blackwell Science; 1998:165-188.
  107. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice 1993;24(2):190-195. [CrossRef]
  108. Lamers F, van Oppen P, Comijs HC, Smit JH, Spinhoven P, van Balkom AJ, et al. Comorbidity patterns of anxiety and depressive disorders in a large cohort study: the Netherlands Study of Depression and Anxiety (NESDA). J Clin Psychiatry 2011 Mar;72(3):341-348. [CrossRef] [Medline]
  109. Jacobi F, Wittchen HU, Holting C, Höfler M, Pfister H, Müller N, et al. Prevalence, co-morbidity and correlates of mental disorders in the general population: results from the German Health Interview and Examination Survey (GHS). Psychol Med 2004 May;34(4):597-611. [CrossRef] [Medline]
  110. van Boeijen CA, van Oppen P, van Balkom AJ, Visser S, Kempe PT, Blankenstein N, et al. Treatment of anxiety disorders in primary care practice: a randomised controlled trial. Br J Gen Pract 2005 Oct;55(519):763-769 [FREE Full text] [Medline]
  111. Longwell BT, Truax P. The differential effects of weekly, monthly, and bimonthly administrations of the beck Depression Inventory-II: Psychometric properties and clinical implications. Behavior Therapy 2005 Jun;36(3):265-275 [FREE Full text] [CrossRef]
  112. Hatzenbuehler LC, Parpal M, Matthews L. Classifying college students as depressed or nondepressed using the Beck Depression Inventory: an empirical analysis. J Consult Clin Psychol 1983 Jun;51(3):360-366. [Medline]
  113. Haaga DA. Introduction to the special section on stepped care models in psychotherapy. J Consult Clin Psychol 2000 Aug;68(4):547-548. [Medline]
  114. Farrand P, Confue P, Byng R, Shaw S. Guided self-help supported by paraprofessional mental health workers: an uncontrolled before--after cohort study. Health Soc Care Community 2009 Feb;17(1):9-17. [CrossRef] [Medline]
  115. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One 2010;5(6):e10939 [FREE Full text] [CrossRef] [Medline]

AUDIT: The Alcohol Use Disorder Identification Test
ANOVA: analysis of variance
ANCOVA: analysis of covariance
ANU: Australian National University
BAI: Beck Anxiety Inventory
BDI-II: Beck Depression Inventory, second edition
CBT: cognitive behavioral therapy
DSM-IV: Diagnostic and Statistical Manual of Mental Disorders, 4th edition
DUDIT: The Drug Use Disorder Identification Test
EQ-5D: EuroQol Group 5-Dimension Self-Report Questionnaire
EQ Index: index score of the EQ-5D
EQ VAS: visual analog scale of the EQ-5D
GP: general practitioner
HADS: Hospital Anxiety and Depression Scale
ICD-10: International Classification of Diseases, 10th edition
ITT: intention-to-treat
SWLS: Satisfaction with Life Scale

Edited by G Eysenbach; submitted 19.05.13; peer-reviewed by T Berger, T Donker; comments to author 10.06.13; revised version received 03.07.13; accepted 12.07.13; published 05.08.13


©Ragnhild Sørensen Høifødt, Kjersti R Lillevoll, Kathleen M Griffiths, Tom Wilsgaard, Martin Eisemann, Knut Waterloo, Nils Kolstrup. Originally published in the Journal of Medical Internet Research (, 05.08.2013.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.