We recruited participants through advertisements about Internet self-help treatment for symptoms of depression, anxiety, and work-related stress placed in local and national newspapers. We aimed at including 200 participants in order to be able to demonstrate moderate effects of d = 0.40 while using a power (1 − β) of 80% and an alpha of .05. We were contacted through email by 299 people (Figure 1). These 299 potential participants received an information booklet and an informed consent form by post as well as a baseline questionnaire through the Internet. All 213 individuals who returned the informed consent and the baseline questionnaire were included. No inclusion or exclusion criteria were used because the intervention was aimed at the general population. Enrollment took place between November 30 and December 20, 2005. The study was approved by the Medical Ethical Committee of the Vrije Universiteit Medical Center, Amsterdam, The Netherlands.

Figure 1

Flowchart of participants

The intervention was Web-based (see Multimedia Appendix for screenshots). Participants were provided with a username and password to access the website. Every week an automated email was sent to the participants to explain the contents and exercises for the coming week. All the information as well as the exercise forms could also be downloaded from the website in case participants preferred to read the information on paper. Master’s level psychology students, trained and supervised by the authors (PC, AvS), offered feedback on the completed exercises. This feedback was not therapeutic but was directed at mastering the proposed problem-solving strategies. For a participant completing the course, the total time spent by the psychology students on feedback was approximately 45 minutes. The course takes 4 weeks.

The intervention consists of three steps:

The second step is the most important of the intervention. For each of the three types of problem (ie, a, b, and c), a different strategy is proposed to cope with it. For the solvable problems (ie, b), we propose the following procedure: (1) write a clear definition of the problem, (2) generate multiple solutions to the problem, (3) select the best solution, (4) work out a systematic plan for this solution, (5) carry out the solution, and (6) evaluate as to whether the solution has resolved the problem.

All participants were randomly assigned to either the self-help course or a waiting list. Questionnaires were sent before the start of the course and 5 weeks later, after the intervention group had finished. Thereafter, the participants in the waiting list group could complete the course.

Randomization took place 1 week before the start of the intervention. We used block randomization with blocks of 10. The randomization scheme was derived by computer and carried out by an independent researcher. All participants were informed by email about the randomization outcome.

Depressive symptoms were measured with the Center for Epidemiological Studies Depression Scale (CES-D) [21] and the Major Depression Inventory (MDI) [22]. The CES-D is a 20-item, self-report questionnaire on feelings of depression; its total score ranges from 0 (no depressive symptoms at all) to 60 (many depressive symptoms). The MDI contains 12 items that are used to calculate the scores on the 10 ICD-10 (International Statistical Classification of Diseases and Related Health Problems, 10th Revision) symptoms of depression. Each of the 10 symptoms is scored on a scale from 0 (at no time) to 5 (all of the time). The total score is calculated by adding all the items, and thus ranges from 0 to 50. Based on the symptom scores, it is also possible to determine the presence or absence of major depression according to the DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, 4th Edition) criteria.

Symptoms of anxiety were measured with the seven anxiety questions of the Hospital Anxiety and Depression Scale (HADS) [23] and the anxiety section of the Symptom Checklist (SCL-A) [24]. The total score of the HADS varies from 0 (no complaints of anxiety) to 21 (many complaints of anxiety). The SCL-A consists of 10 questions, and the total score ranges from 10 (no complaints of anxiety) to 50 (many complaints of anxiety).

Work-related stress was measured with the Dutch version of the Maslach Burnout Inventory (MBI) [25], which contains three subscales: (1) emotional exhaustion (MBI-EE), 5 items; (2) depersonalization (MBI-DP), 4 items; and (3) personal accomplishment (MBI-PA), 6 items. Each item is scored on a scale from 0 to 6, and subscale scores are calculated by adding the item scores and dividing this subscale total score by the number of items. For MBI-EE and MBI-DP, a higher score indicates more work-related stress, while a high MBI-PA score indicates less work-related stress. Individuals can be considered burnt out when they report high MBI-EE (≥ 2.2) in combination with high MBI-DP (≥ 2.0) or low MBI-PA (≤ 3.66) [26].

Quality of life was assessed with the EuroQoL questionnaire (EQ-5D) [27]. The EQ-5D consists of 5 items (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression), each of which is rated as causing “no problems,” “some problems,” or “extreme problems.” The EQ-5D can thus describe 486 unique health states. Each of these health states has been empirically valued between 0 (poor health) and 1 (perfect health). The scores of our respondents were weighted with these values to derive a single summary index score.

Out of 213 enrolled participants, 177 filled in the post-test questionnaires (response rate 83.1%). The response was significantly higher in the control group (91%; n = 96) than in the intervention group (76%, n = 81; P = .004). Furthermore, the response was higher among the more educated participants (94.9%; n = 111) than among less educated participants (69%, n = 66; P < .001) and higher among participants without alcohol problems (87.1%, n = 121) than among those with alcohol problems (76%, n = 56; P = .04).

All the baseline differences between responders and nonresponders on the outcome measures were in the same direction: nonresponders reported poorer health at baseline than responders. However, the differences were very small and not statistically significant (Table 1).

As shown in Table 2, most participants in this study were female (71.4%; n = 152), born in the Netherlands (91.5%; n = 195), higher educated (54.9%; n = 117), and had a paid job (64.8%; n = 138). The participants in the intervention group were more often married (59.8%; n = 64) than participants in the control group (44.3%; n = 47; P = .02). There were no differences between the intervention and control groups with regard to baseline depression, anxiety, stress, or quality of life scores (Table 3).

Of all 107 participants in the intervention group, 9% (n = 10) dropped out before the course started. The first assignment (Week 1) was completed by the remaining 91% (n = 97). Then another 17% (n = 18) dropped out, and the second assignment (Week 2) was completed by 74% (n = 79). Another 8% (n = 9) dropped out, and the third assignment (Week 3) was completed by 65% (n = 70). Finally, another 10% (n = 11) dropped out, leaving 55% (n = 59) who completed the whole course. Married participants more often completed the course (66%; n = 42) than non-married participants (40%, n = 17; P = .008). There were no other significant demographic or baseline differences between the participants who did or did not complete the course.

In general, the intervention had a significant effect on symptoms of depression, anxiety, and quality of life but not on work-related stress (Table 3). The analyses of all participants showed the most profound effects for the CES-D (d = 0.50) and the SCL-A (d = 0.42). In general, the effect sizes were largest for those participants who fully completed the intervention (n = 59). For these, the intervention was most effective for depression (CES-D: d = 0.67; MDI: d = 0.56), but the results for anxiety (SCL-A: d = 0.51; HADS: d = 0.48) and quality of life (EQ-5D: d = 0.44) were also substantial.

In a subset analysis, we selected only the participants with the most severe problems at baseline and calculated their improvements for each measure (Table 4). Compared to all participants (see Table 3), those with the most severe problems at baseline improved more, as evidenced by higher effect sizes, with the exception of scores on the SCL-A scale, for which the effect size decreased from 0.42 to 0.37. Improvements in effect size were most notable for work-related stress: the overall effect size on the MBI-EE subscale was 0.28 for all participants but improved to 0.65 for participants who actually experienced a burnout at baseline.

The proportion of participants with significant improvements (their change is so large it is unlikely to have occurred by chance, see definition under “Methods”) in both groups is compared in Table 5. The results show significant effects of the intervention both for depression (CES-D and MDI) and anxiety (SCL-A and HADS). The improvements on the MBI-PA scale are also statistically significant. The differences between the intervention and the control groups for the remaining outcomes were all in favour of the intervention group (OR between 1.6 and 2.2), but these results were not statistically significant.

Of all 81 participants who suffered major depression according to the MDI at baseline, a total of 52 (64.4%) had recovered at post-test across both groups (Table 6). Recovery occurred more often in the intervention group (72.7%) than in the control group (54.6%, OR = 2.2), but this effect was not statistically significant (95% CI 0.8-6.0). Recovery from anxiety and burnout also occurred more often in the intervention group than in the control group. However, the result with regard to anxiety was not statistically significant (OR = 2.0; 95% CI 0.9-4.2), while that for burnout was (OR = 4.0; 95% CI 1.2-13.5).

We studied the effects of a short, generic, Web-based, self-help intervention for mental health problems in a randomized trial among 213 participants with symptoms of depression, anxiety, or work-related stress. We demonstrated statistically and clinically significant effects on symptoms of depression and anxiety. These effects were even more pronounced among participants with more severe baseline problems and for participants who fully completed the course. The effects on work-related stress and quality of life were less clear.

This study has several limitations. The first is related to the choice of the control group. We could have chosen a care-as-usual comparison (ie, not have given any intervention to the control group); however, this might have limited the generalizability of our results since in that case only patients willing to be randomized to a non-treatment option would have participated. It is likely that these patients differ from the ones who do want (need) treatment. We also might have chosen an attention placebo control group or comparison with another intervention. It is known that effects of attention placebo controlled trials are usually smaller than waitlist controlled trials. However, with our intervention, we especially intended to reach those people who do not get any treatment at all [33], and in this case, we feel that a comparison with a waiting list control group is justified. We stress that the demonstrated effects might, in part, be caused by common therapy factors and not by the specific intervention we studied (eg, the attention given to the intervention group by means of email support might have caused effects regardless of the contents of the feedback or the intervention). Nevertheless, since we intend to implement the course in the Netherlands as is (including support), this effect is what we wanted to measure.

The second limitation has to do with the response rate. Although the overall response rate was satisfactory (83%), the response rate of the intervention group was significantly lower (76%) than that of the control group (91%). We could find no indications for selection bias since we could not demonstrate clear baseline differences between the responders and nonresponders (except for marital status). The bias that still might have been introduced was accounted for by imputing all missing data (multiple imputations) and performing intention-to-treat analyses. Nevertheless, imputing 24% of the data might have led to unreliable estimates.

Another limitation is the fact that participants could only be included in the study if they had computer skills and access to Internet. Thus, the participants in this study were more highly educated than the general population, and it is uncertain whether the results of this study can be generalized to people with less education.

Meta-analyses for bibliotherapy regarding different types of target problems have shown effect sizes between 0.53 and 0.96 [3]. For depression and anxiety, a recent meta-analysis of Web-based, cognitive behavioral, self-help interventions showed mean effect sizes (Cohen’s d) of 0.32 and 0.96, respectively [5]. Thus, our results on symptoms of depression and anxiety seem to fit well within the reported range. It is important to note that our results were obtained in less time (4 weeks) than is usual for Web-based interventions for anxiety or depression (often 6 weeks or more). Furthermore, our results are also almost identical to those found in a meta-analysis of face-to-face problem-solving treatment (d = 0.42) [34]. All this implies that our intervention may be a worthwhile alternative to other more intensive or expensive treatment options, especially since it can be used for participants with comorbid symptoms of anxiety and depression. However, longer follow-up studies are necessary to determine the treatment gains over a longer period of time.

The results with regard to work-related stress were less consistent. When considering only those participants who were suffering from burnout at the start of the study, the results were promising. The participants in the intervention group were four times (95% CI 1.2-13.5) more likely to recover from their burnout than participants in the control group, and they experienced a substantial improvement with regard to the EE subscale of the MBI (Cohen’s d = 0.65). These effects disappeared when considering all participants (or all participants who completed the intervention). This probably can be explained by the relatively small percentage of participants who actually did experience work-related stress at the start of the study: only 77 participants (36%) could be described as suffering from burnout. Furthermore, it must be noted that, in general, the effects of interventions for work-related stress seem to be less pronounced. Meta-analyses have reported effect sizes between 0.35 and 0.68 for different types of face-to-face intervention [35].

To our knowledge, this is the first trial on a short, Web-based, problem-solving intervention for participants with different types of (comorbid) emotional problems. The results seem to be as good as other longer, disease-specific bibliotherapies. Longitudinal research is needed to study the long-term effects.