A two-armed randomized controlled trial (RCT) was conducted that compared the effectiveness of the Web-based CDMIs with a no-intervention wait-listed control group that had unlimited access to usual care. Participants were assessed at three time points: at baseline and at 3- and 6-month follow-ups. The trial is reported in accordance with the CONSORT-EHEALTH checklist V1.6.1 (see Multimedia Appendix 1). The Medical Ethics Committee of the University Medical Center Utrecht approved the study protocol in 2014, and the trial was registered at the Netherlands Trial Register on May 27, 2014 (No: NTR4612).

Participants were eligible for participation if they were adults (≥18 years), had access to a computer with an Internet connection, sufficient proficiency of the Dutch language, adequate computer skills to participate in the training, and mild-to-moderate depressive symptoms defined as a score of 14 to 38 on the Inventory of Depressive Symptomatology Self-Report (IDS-SR) [32,33]. These IDS-SR cut-off scores imply that the CDMIs in this study were used for, at least but not limited to, an indicated preventive purpose. Suicidal thoughts or plans, as measured with item 18 of the IDS-SR, were a reason for exclusion (a score of >1 was used; initially a score of >0 was used, but this was deemed too strict). If participants indicated they had suicidal thoughts or plans, they were advised to contact their GP or an anonymous online platform for people with suicidal thoughts or behaviors (113Online [34]).

Participants were recruited from June 2014 to January 2015 via open recruitment (ie, through relevant websites, messages on social media, and messages in digital newsletters of the Trimbos Institute). People interested in participation were referred to a special study website where they were given more information about the study and could register to take part in the study by completing a written or an online informed consent form including their name and email address. Once informed consent was given, the eligibility criteria were assessed. Applicants were requested to complete the first part of the self-report online baseline questionnaire, which acted as a screening instrument and consisted of the IDS-SR and questions about age, Internet access, and computer skills. People who did not fulfill the inclusion criteria were informed during or immediately after completing the first part of the baseline questionnaire and the reason for exclusion was provided.

Eligible participants received the second part of the online baseline questionnaire. To be randomized, the applicants were required to complete the entire baseline questionnaire. To be able to conduct the stratified block randomization, applicants were asked which CDMI they would want to take part in (“sleep better,” “stress less,” or “worry less”). Randomization to the intervention group or control group occurred automatically using a 1:1 ratio. Directly after randomization, participants allocated to the experimental condition received an email with an activation code for creating an online account that gave them access to the CDMI of their choice. The account was valid for one year. Participants in the control condition were sent an email with the outcome of the randomization, including the message that the CDMI would become accessible to them after 3 months.

The CDMIs are Web-based self-help interventions without therapist guidance [35], developed by the Trimbos Institute, a nonprofit organization. There are three different CDMIs: “sleep better,” “stress less,” and “worry less.” The CDMIs target people with mild-to-moderate symptoms of depression who experience problems with sleep, stress, or worry. In the interventions, users learn to better understand and deal with the problem of their choice. In fact, a unique feature of the CDMIs as an approach for depressive symptoms is that they were developed taking into account that symptoms of depression or a developing depression vary by individual. Therefore, they are complaint-focused rather than disorder-focused, thus participants can choose the CDMI they want to use based on their personal needs and do not have to use CDMIs that are not relevant to their situation. The primary goal of the CDMIs is to reduce or prevent depressive complaints. Secondary goals are to reduce sleep problems, stress, or worry.

The content of the CDMIs is largely based on cognitive behavioral techniques and incorporates elements from solution-focused therapy, mindfulness, and positive psychology. The CDMIs are made up of three to four modules, each module consisting of four to six exercises. Some modules are relevant for sleep, stress, and worry and are, therefore, part of all three CDMIs (eg, the module “relaxation”). Fixed elements in every CDMI are a home page, a diary, a list with the user’s favorite exercises, an exercise book, a to-do list, and a library. A more detailed account of the CDMIs (including screenshots) can be found in Multimedia Appendix 2. To gain access to the CDMIs, an account had to be created by entering a username and email address. Users were free to choose between the modules and exercises and could work independently through the CDMI, without supervision. The advised amount of time to spend on the CDMI was 2 to 3 hours a week for a period of at least 4 weeks.

In case any assistance was needed during the study, a contact form on the website of the CDMIs could be used or participants in the study could email or call one of the researchers. Participants were allowed to use any other type of care in addition to the online CDMIs.

Participants in the control group were placed on a waiting list and received access to the online CDMI of their choice after 3 months (following the 3-month postbaseline assessment). The wait-listed participants were aware of this procedure. Participants in the control group were also free to use any other types of care.

Descriptive statistics were used to describe the characteristics of the study sample at baseline. Attrition analysis was conducted by comparing demographic characteristics and primary and secondary outcome variables of the participants who completed the questionnaire at T1 with those who did not complete the T1 assessment. For this purpose, differences between the groups were tested using independent t tests and chi-square statistics.

Analyses of the effectiveness of the CDMIs were carried out according to the intention-to-treat principle. Linear mixed models were used to estimate the effects of the CDMIs on the primary and secondary outcomes. This technique allows for the correlation between longitudinal data and uses all available data points, thus not discarding cases due to a missing value. Missing values are accounted for using the maximum likelihood method to estimate coefficients. A random intercept was fitted with an identity covariance structure. Time was defined as the within-group factor and the study condition (CDMI or wait-list) as the between-groups factor. The mean difference in the outcomes between the study conditions over time is expressed by the condition×time interaction.

The 6-month follow-up data were analyzed separately for the intervention and control group (ie, only within-group changes analyzed) because the control group had gained access to the CDMI intervention by that time. Growth curves were examined to determine whether any effects in the intervention group remained or increased at 6 months (ie, no significant decrease in effect between T1 and T2) and whether effects increased in the control group after the 3-month follow-up (ie, significant increase in effect between T1 and T2). An exploratory analysis was undertaken into the relationship between intervention use (number of log-ins) and effectiveness within the intervention group during the 6-month intervention period. To this end, the interaction between use and time was tested in a linear mixed model.

All analyses were adjusted for baseline demographic factors (age, gender, living arrangement, and education level). Adjustments were also made for factors that were related to dropout as indicated by the attrition analysis. Standardized effect sizes (Cohen d) were calculated using the estimated (adjusted) mean differences of the outcomes from the linear mixed models divided by the pooled standard deviation of the outcomes at baseline. Analyses were performed using SPSS version 22 and Stata version 12.1 statistical software packages.

In total, 525 people showed interest in the study by giving their informed consent. These individuals were invited to complete the first (screening) part of the baseline questionnaire. Sixty-three people did not complete the baseline questionnaire and were excluded from the study. Another 133 people did not meet the inclusion criteria, primarily because of too few or too many depressive symptoms or because of the presence of suicidal thoughts or behaviors. This resulted in a study population of 329 participants, of which 165 were randomized to the experimental condition and 164 to the control condition. Of the 165 participants in the experimental condition, 59 participated in the sleep CDMI, 45 participated in the stress CDMI, and 61 participated in the worry CDMI. Figure 1 shows the flow of participants during the trial.

Table 1 shows an overview of the baseline characteristics of the participants. The participants had a mean age of 43 (SD 12.93, range 18-81) years and the majority were female (75.7%, 249/329). More than 98% (323/329) had Dutch nationality, and a majority had a paid job (70.8%, 233/329), a high education level (72.0%, 237/329), and a high income (70.2%, 231/329). Participants in the control condition rated the experienced severity of their complaints as high less often (49.4%, 81/164) than the participants in the intervention condition (58.8%, 97/165).

At T1, data were available for 237 participants (dropout rate 28.0%, 92/329). The loss of participants at T1 was significantly higher in the experimental condition (41.2%, 68/165) compared to the control condition (14.6%, 24/164; χ²₁=28.8, P<.001). Analysis of baseline factors showed there was a significant association between baseline anxiety scores and attrition. The mean score on the GAD-7 at T0 was 1.2 points lower in participants who completed the T1 assessment as compared to those who did not (t₃₂₇=–2.45, P=.02). Consequently, we adjusted for this variable in all analyses. Ten participants indicated suicidal thoughts or plans at T1, of which eight belonged to the control condition and hence did not gain access to the CDMIs.

At T2, 150 participants completed the questionnaire (dropout rate 54.4%, 179/329). Again, attrition was higher in the experimental condition (58.8%, 97/165) than in the control condition (50.0%, 82/164), but this difference was not significant. Suicidal thoughts or plans were indicated by seven participants at T2: five from the experimental condition and two from the control condition.

The observed and estimated marginal means (estimated means adjusted for all factors in the model) for all outcomes at baseline and 3-month follow-up are presented in Table 2. The results of the linear mixed models analysis showed significant time×intervention effects for all outcomes except stress (see Table 3). This means that greater reductions in depression, sleep problems, worry, and anxiety were detected between baseline and the 3-month follow-up in the intervention group compared to the control group. Moreover, greater improvements in well-being were observed over time in the intervention group. The intervention did not have a significant effect on stress complaints. The standardized effect size for the primary outcome of depression was large (Cohen d=–0.70). The magnitude of the effect sizes for the secondary outcomes were generally in the small-to-medium range across the outcomes (Cohen d=–0.20 to 0.40).

Figure 2 shows the course of the (estimated) primary and secondary outcomes in the two groups. In the intervention group, the greatest effects were observed between baseline and T1. These effects did not significantly increase or decrease at the 6-month follow-up, except for further reductions in sleep complaints (see Table 4). Generally, effects in the intervention group were sustained until 6 months. This pattern was reversed in the control group because they did not receive the intervention until after the T1 assessment. The greatest effects were observed between T1 and T2 in the control group, and were of similar magnitude as those found between T0 and T1 in the intervention group (see Table 4). However, there were some exceptions. Stress complaints showed a similar pattern of change in both groups: the greatest reduction between T1 and T0 (which is also reflected in the nonsignificant difference in the effectiveness analysis, see previous), and a smaller reduction between T1 and T2. Moreover, anxiety complaints also seemed to decrease a little more between T0 and T1 than between T1 and T2 in the control group.

Approximately three-quarters of the intervention group actually created an account (127/165, 77.0%); of these participants, 91.3% (116/127) logged into their chosen CDMI at least once. During the 3-month intervention period, the participants logged in a median 3 times (range 0-166, IQR=5). Approximately half of the control group actually created an account (86/164, 52.4%) after completing the T1 measurement; of these participants, 85% (73/86) logged into their chosen CDMI at least once. During the 3-month period between T1 and T2, the participants in the control group logged in a median 2 times (range 0-85, IQR=4). Of the 75 participants in the intervention group who completed the use and satisfaction questions, a majority (56%, 42/75) reported that they spent an average of 30 minutes or more a week on the CDMIs and 8% (6/75) of participants spent 2 hours or more a week on the CDMIs. Of the 48 participants in the control group who completed the use and satisfaction questions, a majority (54%, 26/48) reported that they spent an average of 30 minutes or more a week on the CDMIs and 10% (5/48) spent 2 hours or more a week on the CDMIs.

Exploratory analysis into the relationship between log-ins and effectiveness showed no discernible dose-response relationship, and this was underlined by the finding that there were no significant interaction effects between the number of log-ins and time (see estimated marginal means and test statistics in Multimedia Appendix 3). However, although the overall tests of the interaction terms across the complaints were not significant, there were a number of significant contrasts. Participants who logged in four or more times showed greater decreases in depressive complaints between baseline and 3-month follow-up than those who did not log in (t₂₂₆=–2.12, P=.04). Those who logged in one to three times showed greater decreases in worry complaints between baseline and 3-month follow-up than those who did not log in (t₁₉₁=–1.98, P=.049).

Overall satisfaction with the intervention was moderate to high. The median satisfaction score given by the intervention group (n=75) was 7 (on a scale of 1 to 10) and the mean was 6.3 (SD 1.8). Participants in the control group who accessed the intervention gave a near identical satisfaction rating (n=45; mean 6.4, SD 1.5). Figure 3 shows the ratings participants in the intervention group scored on the various satisfaction items. These results show a similar pattern of moderate to high satisfaction across the various topics.

Various suggestions were made for future improvements. Among the most mentioned were to include reminders, additional information, and more structure to exercises.

This RCT evaluated the effectiveness of unguided Web-based self-help interventions that aimed to decrease depressive complaints by targeting stress, sleep problems, or worry. The results at 3-month follow-up showed a significant reduction in depressive symptoms for participants in the intervention group compared to participants in the wait-list control group. Furthermore, significant effects were observed for sleep problems, worry, anxiety, and well-being. The CDMIs did not significantly reduce stress complaints, but reductions were seen in both the intervention and control groups. At 6-month follow-up, the improvements in the intervention group were generally sustained. The control group, which started with the CDMIs after 3 months, experienced significant improvements at the 6-month follow-up compared to the 3-month follow-up for all outcome variables but stress. In general, participants were moderately to highly satisfied with the CDMIs.

Online self-help interventions targeting symptoms of depression and the prevention of cases of depressive disorder have been found to be effective in previous studies [9,28,49]. A recent meta-analysis found that online interventions for the prevention of depression can reduce depressive symptoms in the short and medium term compared to no-intervention control conditions (eg, short-term pooled effect size of Hedges g=–0.35, 95% CI –0.57 to –0.12) [28]. A meta-analysis by Davies and colleagues [49] also showed that computer-delivered and Web-based interventions can be effective in improving depression in university students, with a pooled standardized mean difference of Hedges g=–0.43 (95% CI –0.63 to –0.22, P<.001). Zhou and colleagues [50] conducted a meta-analysis to the efficacy of Internet-based cognitive behavioral therapy (CBT) for people who fulfill the criteria for at least subthreshold depression. They concluded that in the short-term the efficacy of Internet-based CBT is superior to a nonactive control group (pooled standardized mean difference of –0.28, 95% CI −0.42 to −0.14), whereas the long-term efficacy is inconclusive. Moreover, a recently published meta-analysis about the effectiveness of online mindfulness-based interventions reported a small effect size on depressive symptoms (Hedges g=0.29, 95% CI 0.13-0.46, P=.001) [41].

The effect size for depressive symptoms found in our study (Cohen d=–0.70) is somewhat larger than reported in these meta-analyses. In some cases, this may be due to more restrictive eligibility criteria that were applied in these meta-analyses, including diagnosis and exclusion of depressive disorders at trial entry (eg, [28]), compared to our study. However, other online, unguided psychological interventions for participants with increased levels of depressive symptomatology also showed lower effect sizes [51].

First, we encountered a high dropout rate of participants during the course of the study. High dropout rates are a problem often encountered in eHealth trials [52,53], with rates as high as 50% to 72% reported. Dropout rates tend to be higher for unguided versus guided eHealth trials [53]. Dropout also tends to be higher in intervention groups than control groups as was the case in our study. Although this is not an uncommon occurrence in eHealth studies, it may nonetheless give rise to attrition bias. However, analyses showed that the only measured difference between participants who completed the T1 assessment versus those who did not was the mean score on the GAD-7 (anxiety), which was adjusted for in the analyses. Furthermore, in the analyses, all participants were included according to the intention-to-treat principle and missing data was accounted for by using linear mixed models (ie, maximum likelihood), which is a recommended strategy for estimating unbiased treatment effects [54].

Second, the CDMIs were developed as a preventive approach to depression, but the diagnostic status of the participants in this study was not assessed with a clinical interview because it was not feasible. To avoid being too restrictive and denying people access to the CDMIs who might benefit from the interventions, we included people with mild and moderate depressive symptoms in our trial. We cannot rule out that some participants in the trial might have fulfilled diagnostic criteria for depression at trial entry. Therefore, the findings and effect sizes should be interpreted in light of this fact.

Third, the participants could not be blinded. The reduction in complaints observed by the control group at T2 versus T1 were comparable to the reduction seen in the intervention group at T1 versus T0. Therefore, it is likely that the effects found are not largely influenced by the lack of blinding.

Another point of consideration is the limited use of the intervention, despite the positive effects. The use of the CDMIs was slightly more limited in the control group. This might have been due to a reduction in complaints in the control condition in the first 3 months of the study, reducing the need for the control condition to use the CDMIs once access was provided. The evaluation of the CDMIs by the users does suggest there is room for improvement of the intervention. Adding reminders was a frequently given suggestion for improvement. However, it would be worthwhile to further question users about strategies that may increase the use of the intervention. We conducted some exploratory analyses to examine the relationship between the number of log-ins and effectiveness, which showed no discernible dose-response relationship. However, it is important to interpret these exploratory findings with caution due to potential selection bias. Moreover, different operationalizations of adherence may be differentially related to effectiveness in eHealth research [55], which would be interesting to examine in future research.

Fifth, although our trial applied an open recruitment strategy, meaning that there were no restrictions for participation other than the inclusion criteria, it is possible that potential participants were missed due to the use of mainly Internet-based recruitment avenues. On the other hand, people who do not use the Internet might be less open to using an unguided Web-based self-help intervention. Nevertheless, the participants of the trial might not have truly reflected the general Dutch adult population with access to the Internet and sufficient computer skills.

Finally, the study population consisted of mainly highly educated female participants. Therefore, the generalizability of the findings with respect to males and other educational levels remains to be determined. Moreover, as mentioned in the Introduction, the CDMIs were developed to also suit a range of target groups, including the low SES population, but participation in the CDMIs and in this study was not restricted to individuals with a low SES. The baseline characteristics indicate that 72% of the participants had a high educational level. This means that without specific efforts aimed at inclusion of low SES people, the usage of the intervention among this group will be relatively low.

Integration of the CDMIs in primary care may be a useful next step for several reasons. As mentioned in the Introduction, patients frequently present to their GP with psychological stress, sleep problems, and worry [17-21]. Moreover, the guidelines of the Dutch College of General Practitioners recommend e-mental health as first step interventions for patients with depressive complaints. During and after the trial, we received several requests from GPs and GP nurses for information about and the use of the CDMIs. As such, the CDMIs seem to be suitable interventions for use in the GP setting that satisfy a demand. When implementing the interventions in the GP setting, the CDMIs can also be offered with some guidance from the GP nurse, which might be a way of boosting adherence and effectiveness [30,31]. Guidance may also be needed to increase the reach to low SES populations.

In addition to human guidance, technology itself may offer opportunities for support through persuasive system design features [56], which may be just as effective as human support [57]. Prompts are an example of a feature that may increase adherence [58]. Based on user feedback, we have recently developed an app to supplement the CDMIs by providing motivational quotes, tips, and prompts to use the CDMIs. Recent advances in mHealth provide opportunities for systems to learn about their user and offer meaningful personalized interventions when they are necessary and, thus, potentially offering more sophisticated technology-based support with or without human support (eg, [59]). It is likely that such personalized interventions will contribute to adherence and ultimately to effectiveness, and it would be interesting to explore this avenue more in future research.

A focus on research questions addressing the use and mechanisms of change in online interventions would be a fruitful next step. This is also in line with calls to pay more attention to the evaluation of intervention components rather than intervention packages in alternative research designs to traditional RCTs as a means to better tie in with the rapid and flexible nature of e-mental health development [60,61]. With respect to the CDMIs, as noted by one of the anonymous reviewers of this paper, it may be worthwhile in the future to investigate the effect of user choice. To this end, a comparison could be made between persons who are able to choose their modules and exercises versus those who are restricted to a more traditional approach in which participants have to go through modules sequentially. Also gaining more insight into the optimal amount of time, exercises, or modules that are needed, and the relationship with user characteristics would be valuable to better personalize interventions in the future. Currently, participants of the CDMIs are advised to spend 2 to 3 hours a week for a period of at least 4 weeks. However, a recent study of Bunge et al [62] showed that even a brief, unsupported Internet intervention of 5 to 10 minutes improved depression scores at 1-week follow-up. Finally, future research should focus on gaining insights into the most optimal strategies for reaching and engaging low SES populations in (preventive) interventions targeting psychological complaints, such as the CDMIs.

In conclusion, the online CDMIs were successful in significantly reducing depressive complaints and had an effect on, albeit to a lesser extent, sleep problems, worry, anxiety, and well-being. The intervention did not significantly reduce stress complaints. This study shows that a purely online self-help intervention can have a positive impact on mental health outcomes. Future research should focus on which specific strategies may boost adherence, and increase the reach of the CDMIs among people with low SES.