This study was a randomized controlled trial with 2 arms: a Web-based guided self-help course called Happy@Work and a CAU group. The full design of the study has been described in detail elsewhere [32].

The sample size was guided by the expected difference in the primary outcome (ie, depressive symptoms) between the 2 groups. Based on a power of 0.80, an alpha of.05, and an expected dropout percentage of 30%, we would need 100 participants in each condition to be able to show an effect-size Cohen’s d of 0.50. Therefore, the total sample size was determined at 200.

Randomization took place at an individual level after completion of the baseline measurement (questionnaire and clinical interview). We used stratification at 2 levels: (1) use of antidepressants and (2) receiving treatment from a psychologist or psychiatrist at study entrance. Block randomization was used with random blocks containing 4, 6, or 8 allocations. An independent researcher made the allocation schedule with a computerized random number generator and the investigators had no knowledge of the schedule. Participants were randomized into 2 groups: the intervention group or the CAU group. Participants were informed about the randomization outcome via email.

The intervention Happy@Work [33] is a brief Web-based intervention delivered with minimal guidance. It consists of 2 evidence-based treatments: problem-solving treatment (PST) [34], cognitive therapy [35], and a guideline for employees to help them to prevent work-related stress [36,37]. In PST, it is assumed that depressive symptoms can be caused by practical problems that people face in their daily lives. It is believed that when people can resolve their problems, their symptoms of depression will decrease [38]. The PST will help them solve their problems. Sometimes, however, problem solving can be disrupted by automatic thoughts such as “I am too weak to solve this problem” or “I will fail solving this problem.” PST may not be sufficient to change these automatic thoughts that disrupt problem solving. Therefore, we incorporated cognitive therapy information and assignments to change these automatic thoughts in the course [35]. Some of the problems that people face are likely to be work-related. These problems are sometimes more difficult for people to comprehend [36,37]. Therefore, one lesson is focused on work-related problems specifically. Happy@Work primarily focuses on the depressive symptoms of the employee, but also incorporates psychoeducation and assignments related to dealing with stress and burnout symptoms because there is a clear relationship between the different constructs [29,39-42]. The intervention consists of 6 weekly lessons with an option of 1 week extra time in case of delay. Each lesson has a different theme, but always follows the same structure: information about the theme, examples, and assignments. Themes of the lessons are introduction of problem solving (lesson 1), problem-solving methods (lesson 2), changing cognitions (lesson 3), dealing with work-related problems (lesson 4), social support (lesson 5), and relapse prevention (lesson 6). Screenshots of the intervention can be found in Figure 1 and Multimedia Appendix 1.

At the start of the intervention, an account was generated by the researchers on the website and a coach was assigned to the participant on the website. Once the account was generated, an automatic email was sent to the participant with a link to activate the account. Participants used their email address and a self-created password to log in once the account was activated.

Participants were asked to submit their weekly assignment via the website after completion and subsequently they received feedback from a coach, again via the website, within 3 working days. Next, automatic emails were sent to participants to notify them about the feedback, to describe the following lesson, and to give the deadline for completion of the next assignment. Participants were able to start with a new lesson after they had received the feedback (ie tunneled intervention). When deadlines were not met, email reminders were sent to participants. There were no content changes, bug fixes, or periods of downtime required during the trial.

The coaches were Master’s students in clinical psychology with training of 6 hours. All coaches used a protocol-treatment manual throughout the course. To ensure treatment fidelity, all feedback was reviewed by a supervisor (AG) before it was placed on the website. The support includes feedback on the assignments and motivational and empathic strategies to keep participants engaged in the course. Development of a patient-therapist alliance, as in traditional psychotherapy, was not an aim of the support.

Participants in the CAU group did not receive treatment or support from the coaches. However, in the email with the randomization outcome they were advised to consult their (occupational) physician or a psychologist if they wanted treatment for their depressive symptoms. Participants who were interested were sent a copy of the self-help book version of the intervention after having completed the posttreatment assessment. Participants in both conditions had access to any additional (mental) health care.

Participants filled in several questionnaires at baseline (T0) and 8 weeks later (T1). Both assessments took place online. Participants also participated in a clinical interview at T0 via telephone.

Symptoms of depression were measured with the Center for Epidemiological Studies Depression scale (CES-D) [43]. This questionnaire is widely used for identifying people with depressive symptoms. Its validity has been tested in different populations [44-46]. The CES-D consists of 20 items and the total score varies between 0 and 60. The Cronbach alpha in this study was .82. A score of 16 or higher represents a clinically significant level of depressive symptoms [43]. The cut-off score of 16 was used in this study as an inclusion criterion.

The anxiety subscale of the Hospital Anxiety and Depression Scale (HADS) was used to measure anxiety symptoms [47]. The anxiety subscale of the HADS consists of 7 items. Scores range from 0 to 21, with higher scores indicating more anxiety. The HADS has shown good homogeneity and reliability in different normal and clinical Dutch samples [48]. The Cronbach alpha in this study was .76.

Burnout symptoms were measured with the Dutch version of the Maslach Burnout Inventory-General Scale (MBI) [49,50]. This self-report questionnaire contains 15 items and 3 dimensions: exhaustion (5 items), cynicism (4 items), and reduced professional efficacy (6 items). Every item was scored on a 7-point Likert scale (0-6). Following the manual of the questionnaire [50], a total score for every dimension was calculated by adding the item scores and dividing that total score by the number of items, with higher scores indicating more severe symptoms. Participants with a high score on exhaustion and a high score on cynicism or a high score on reduced professional efficacy were considered as “burnout” [50]. We rescored the professional efficacy dimension with higher scores indicating less feeling of professional efficacy, hence the high score in the burnout diagnoses. The Cronbach alphas for the different dimensions in this study were .83 for exhaustion, .83 for cynicism, and .79 for reduced professional efficacy.

We used the general work performance scale of the World Health Organization (WHO) Health and Work Performance Questionnaire (HPQ) [51], which contains 4 items. Item 4 gives the best and easiest indication of the subject’s perception of their own work performance [52] and we report that item only in this study. On item 4, participants were asked to rate their overall work performance during the past 4 weeks when compared to employees with comparable functions. It was scored on a 7-point Likert scale with a higher score indicating poorer work performance compared to other employees [52].

The WHO Composite International Diagnostic Interview version 2.1 (CIDI) [53] is a structured interview to assess psychiatric diagnosis defined in the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders, 4th edition, Text Revision (DSM-IV-TR) [54]. For this study, 2 sections of the CIDI were assessed: the mood disorders section and the “other” anxiety disorders (social phobia, panic disorder, agoraphobia, and generalized anxiety disorder) section. The CIDI was conducted by trained interviewers via telephone at baseline (T0) and was used for diagnostic purposes.

A revised version of the Trimbos and iMTA Questionnaire on Costs Associated with Psychiatric Illness (TiC-P) [55] was used to collect data on health care utilization. The TiC-P is a self-report questionnaire and consists of 2 different parts that can be administrated separately. Part I was used and contains 12 items concerning health care utilization by participants. There were 2 questions added to the questionnaire on the frequency of utilization of different health care services of the company: occupational physician and occupational social work. The questionnaire was used at T0 to assess health care utilization up to 3 months before the start of the study and at posttreatment (T1) assessment to assess health care utilization between baseline and posttreatment assessment.

We added demographic questions, working hours, medication use for psychological problems, and treatment by a mental health specialist to the baseline questionnaire.

Baseline differences in demographic variables and outcome measures were investigated using chi-square tests and independent-sample t tests.

Baseline data were available for all participants. Missing values at posttreatment (T1) (26.0%, 171/231) were handled by multiple imputation using the fully conditioned specified method with model type predictive mean matching in SPSS version 20.0 (IBM Corp, Armonk, NY, USA) creating 5 datasets. In multiple imputation, missing data are imputed by regression analyses and the available baseline data (demographics and baseline scores on outcome measurements) of the study completers as well as the study dropouts at posttreatment are used to estimate missing values at posttreatment. This provides a more reliable estimation of the “real” data than other imputation methods. The analyses were performed on the 5 created datasets and combined into a single overall estimate using the multiple imputation inference rules of Rubin [56]. This yields proper P values and confidence intervals for the estimates.

Primary effectiveness analyses were conducted according to the intention-to-treat (ITT) principle. We calculated the intraclass correlation (ICC) to examine nonindependence of observations at the company level. The results showed that this was not an issue (ICC=.02, P=.77); therefore, a multilevel approach for analyzing the data was not deemed necessary. We performed linear regression analyses to determine differences between the intervention group and the CAU group with the posttreatment score as the dependent variable and group (intervention or CAU) as the predictor variable while controlling for baseline scores for every outcome measure. The magnitude of the effect is expressed in Cohen’s d [57]. The Cohen’s d was calculated by subtracting the post treatment mean score of the CAU group of the posttreatment mean score of the intervention group and dividing that result by the pooled standard deviation. Effect sizes of ≥0.8 are assumed to be large, effect sizes of 0.5-0.8 are moderate, and effect sizes of 0.2-0.5 are assumed to be small [57]. We calculated the effect sizes for all participants (ITT) and for participants who completed the intervention (per protocol analysis). Completion of the intervention was defined as completion of ≥5 lessons of the intervention. Reporting both outcomes is of high importance because intervention dropout is common in Web-based interventions.

We calculated clinical significant change as described by Jacobson and Truax [58]. This method uses a reliable change index as an index for improvement and its formula is pretest score minus posttest score divided by the standard error of the difference between the 2 test scores. To calculate the standard error of the difference between the 2 test scores in the formula, one uses the pretest SDs of the outcome and the reliability of the questionnaire. In this study, we used the following reliability scores from the questionnaires: 0.90 for the CES-D [59], 0.83 for the HADS [60], 0.88 for the exhaustion scale of MBI [50], 0.81 for the cynicism scale of MBI [50], and 0.75 for the reduced professional efficacy scale of MBI [50]. If the outcome of the sum of the reliable change index is greater than 1.96, this is considered a significant improvement because the amount of change is unlikely to have occurred by chance. The differences in clinical significant improvement rates were then expressed as odds ratios (ORs).

We calculated the recovery rates for depressive symptoms in which a score <16 on the CES-D at posttreatment was defined as recovery [59]. The differences in recovery rates were then expressed as ORs. Finally, we defined reliable recovery as clinical significant improvement between baseline and posttreatment score and recovery at posttreatment. Reliable recovery was also expressed as OR. Results are presented as the mean and standard deviations of the observed data. Between- and within-group effect sizes as well as effectiveness analyses were based on the pooled results of the imputed data.

Figure 2 shows the flow of participants through the trial. A total of 778 employees were interested in the study and applied for more information; 320 of them did not return the informed consent form and/or did not fill in the baseline questionnaire. We received a completed informed consent form and a baseline questionnaire from 458 participants. Of those, 208 were not eligible to take part in the study because they scored below the cut-off score of 16 on the CES-D (n=144), were already absent from work (n=48), had unstable medication use for depressive symptoms (n=12), or had a legal labor dispute with their employer (n=4). Further, 17 participants did not complete the diagnostic interview and 2 participants withdrew from the study before randomization. The remaining 231 participants were randomized to either the intervention group (n=116) or the CAU group (n=115). Most participants (n=166) were employed by 1 of the 2 banking companies, 39 by the research institutes, 11 by the security company, and 15 by the university. Of the 231 participants, 10 (4.3%) used medication without psychological treatment, 24 (10.4%) received psychological treatment but no medication, and 4 participants (1.7%) used both medication and received psychological treatment at baseline. Thus, most participants (83.6%) did not receive treatment for their depressive symptoms.

As shown in Table 1, most participants were female (62.3%, 144/231), born in the Netherlands (95.2%, 220/231), involved in an intimate relationship (76.2%, 176/231), highly educated (63.6%, 147/231), and worked for 34 hours per week on average. Participants in the intervention group were more often born outside the Netherlands (7.8%, 9/116) than participants in the CAU group (1.7%, 2/115, P=.03). There were no significant differences between the intervention group and the CAU group on any of the outcome measures at baseline.

A total of 57 participants (24.7%) were diagnosed with a current major depressive disorder, dysthymic disorder, or both; 23 participants in the intervention group and 34 in the CAU group. Of those, 15 participants suffered from a single episode major depressive disorder (9 intervention, 6 CAU), 40 participants from a recurrent major depressive disorder (12 intervention, 28 CAU), and 9 participants from a dysthymic disorder (7 intervention, 2 CAU).

Anxiety disorders were less frequently present; a total of 48 participants (20.8%) were diagnosed, 27 participants in the intervention group and 21 in the CAU group. The different anxiety disorders frequencies were as follow: social phobia (14 intervention, 7 CAU), panic disorder without agoraphobia (2 intervention, 0 CAU), panic disorder with agoraphobia (4 intervention, 2 CAU), or generalized anxiety (11 intervention, 16 CAU) disorder.

At posttreatment, we analyzed the health care utilization of both groups to get a more detailed view on health care utilization by the CAU group. Only a small number of the total participants made use of health care. More participants in the CAU group sought help in general health care (19 intervention, 25 CAU) or occupational health care (4 intervention, 7 CAU), but this was not statistically significant compared to the intervention group (t ₄₃=0.35, P=.73; t ₉=1.09, P=.31). There was only a slight difference in medication use for depressive symptoms between the groups (8 intervention, 10 CAU), and this was not statistically significant different (χ² ₁=0.00, P=.96).

A total of 61 participants (26.4%) did not complete the T1 (posttreatment) assessment: 42/116 (36.2%) of the intervention group and 19/115 (16.5%) of the CAU group. Data from 173 of 231 participants (74.9%) was available for the primary outcome depressive symptoms, and data from 171of 231 participants (74.0%) was available for the other outcomes. Participants in the CAU group more often completed the T1 assessment (χ² ₁=11.5, P=.001). Attrition rates were also lower for participants who completed the intervention (χ² ₁=32.1, P<.001).

Of the participants randomized to the intervention group, 9.5% (11/116) did not start or complete the first lesson of Happy@Work. Lesson 1 was completed by 90.5% (105/116), lesson 2 by 75% (87/116), lesson 3 by 57.8% (67/116), lesson 4 by 49.1% (57/116), lesson 5 by 38.8% (45/116), and lesson 6 by 26.7% (32/116). A total 29 of 116 participants dropped out of the intervention, the other participants were not able to complete more lessons within the time limit of 7 weeks. Most participants who dropped out did not report a reason for dropout. When reasons were reported (14/116) they pertained mostly to lack of time.

All participants improved between baseline and posttreatment on all outcomes measured (see Table 2). For the primary outcome, depressive symptoms, a high within-group effect-size Cohen’s d was found for both the intervention (d=1.03, 95% CI 0.76-1.30, P=.001) and the CAU group (d=0.98, 95% CI 0.71-1.25, P<.001). However, there was no difference between both groups (d=0.16, 95% CI –0.10 to 0.41, P=.29). The same result was found when we compared course completers (n=45) with CAU (d=0.29, 95% CI –0.05 to 0.64, P=.13).

Small to medium within-group effects were found for the secondary outcomes anxiety, burnout, and work performance in both the intervention group and the CAU group (see Table 2). Between-group differences were small for all secondary outcomes, but participants in the intervention group improved significantly more on anxiety symptoms (d=0.16, 95% CI –0.09 to 0.42, P=.04) and the exhaustion dimension of the MBI (d=0.17, 95% CI –0.09 to 0.43, P=.02) compared to CAU. Course completers also improved more on anxiety symptoms compared to CAU (d=0.19, 95% CI –0.16 to 0.53, P=.04), but not on the exhaustion dimension of the MBI (d=0.17, 95% CI –0.18 to 0.52, P=.14). No significant between-group differences were found on the cynicism dimension of the MBI, the reduced professional efficacy dimension of the MBI, and work performance.

Data on clinical significant improvement are reported in Table 3. Clinical significant improvement on depressive symptoms was comparable in both groups (OR 0.9, 95% CI 0.5-1.6, P=.82). More participants in the intervention group showed clinical significant improvement on anxiety symptoms, the exhaustion dimension of the MBI, and the cynicism dimension of the MBI compared to the CAU group, but differences between the groups were not significant (see Table 3). A total of 105 of the 231 participants (45.5%) were recovered from depression at posttreatment. More participants in the intervention group (56/116, 48.3%) recovered from depression compared to the CAU group (49/115, 42.6%), but not significantly (OR 1.3, 95% CI 0.7-2.3, P=.41). Reliable recovery rates for depression were also in favor for the intervention group, with an odds ratio of 1.3. In the intervention group, 44.8% (52/116) reliably recovered and 39.1% (45/115) in the CAU group. However, the difference was not statistically significant (OR 1.3, 95% CI 0.7-2.3, P=.39).