We considered RCTs from 1990 to May 2016 in which adult participants (older than 18 years) experienced stress and were participating in the trial to decrease their stress levels. The search was initially conducted in August 2013 and repeated in May 2016 to ensure it was as current as possible. Studies prior to 1990 were excluded; due to the rapidly changing technology in this field, these programs cannot be compared to the current interventions that are likely to be delivered to participants seeking help. The primary intervention target of included studies needed to be a reduction of stress within the target group. We excluded trials that targeted participants with medical conditions (eg, cancer, tinnitus, headache); caregivers of people with medical conditions (eg, caregivers for dementia); and participants with psychiatric disorders (eg, depression, anxiety or posttraumatic stress disorder), posttraumatic symptoms, postpartum emotional distress, or bereavement. The studies included needed to compare an intervention to any type of comparison group. There were no restrictions with regard to dosage or intensity, delivery, duration, frequency or timing of delivery, or the type of delivery channel (eg, email, Web-based, computerized). Trials investigating stress management as merely one part of the intervention (eg, alongside depression or anxiety) were included only if the primary goal of the intervention was to reduce stress. Furthermore, trials conducted in the context of the promotion of well-being rather than stress reduction were excluded. The studies had to report at least one instrument that claimed to measure stress levels. For the main analysis, we considered postintervention data. We also included follow-up data (when available) to assess longer-term effectiveness.

The search strategy for this meta-analysis was created with four categories of search terms. We defined the search terms to meet the following criteria: (1) stress reduction, (2) evaluation of an intervention program, (3) application of a RCT as an experimental design, and (4) delivery in a Web- or computer-based context. A detailed description of the search terms can be found in the Multimedia Appendix 1.

The specified search strategy was applied in three major database search engines (PsycINFO, PubMed, and Cochrane). Additionally, manual searches in key journals (eg, International Journal of Stress Management, Journal of Medical Internet Research, Journal of Occupational Health, Scandinavian Journal of Work, Environment and Health) and in the reference lists of the included studies were conducted.

After removing duplicates of the articles identified, two researchers (EH and SN) independently screened all titles and abstracts based on the inclusion and exclusion criteria, and two researchers (EH and DL) assessed the full-text articles for eligibility. The researchers who assessed the relevance of the studies were not blind to the authors, institutions, journal of publication, or results.

Data were extracted concerning the origin, the number of participants, age, gender, comparison group, outcomes, theoretical basis, length, guidance, the risk of bias, and follow-up data. In cases of insufficient description, the primary investigators of the respective studies were contacted to obtain missing information.

Two researchers (EH and DL) assessed the risk of bias of the included studies in accordance with the Cochrane Guidelines [40]. Thereby, sequence generation, allocation concealment, blinding, incomplete outcome data, and selective outcome reporting were judged.

Because we expected only a limited number of studies with relatively small sample sizes, we conducted a power calculation to examine how many studies with how many participants would need to be included to assure sufficient statistical power to identify relevant effects. This power calculation was conducted according to the procedures described by Borenstein and colleagues [41]. We hoped to find a sufficient number of studies to be able to identify a small-to-moderate effect size of 0.35. These calculations indicated that we would need to include at least five studies with a mean sample size of 100 (50 participants per condition) to be able to detect an effect size of Cohen d=0.35 (conservatively assuming a medium level of between-study variance, T², a statistical power of .80, and a significance level, alpha, of .05). Alternatively, we would need three studies with 150 participants each to detect an effect size of Cohen d=0.35 or nine studies with 50 participants.

Analyses were conducted using the statistical software Comprehensive Meta-Analysis (version 2.2.057). The effect size of subjective level of stress was calculated as the primary outcome. Furthermore, we assessed the levels of depression and anxiety when available. If more than one measure per outcome was used and if no primary outcome was indicated, then the mean of the effect size was calculated to ensure that each study yielded only one effect size. A random-effects model was chosen because of the expectations of considerable heterogeneity between studies. We further conducted a series of subgroup analyses according to the mixed-effects model. In this model, studies within subgroups are pooled with the random-effects model, whereas tests for significant differences between subgroups are conducted with the fixed-effects model. Because of the small number of studies providing follow-up data, the subgroup analyses were performed only for posttreatment and the results from the follow-ups for the primary outcome stress were clustered into two categories (1-3 months and 4-6 months).

In three studies, two treatments were compared with a single control group [36,38,42]. In these cases, we treated each comparison as a separate study, and we avoided the double counting of controls by dividing the control group in half.

The effect size in the form of Cohen d was used to represent the standard mean difference between the means of the intervention group and the control group at posttest. According to Cohen [43], d=0.2 can be considered a small effect, d=0.5 is a medium effect, and d=0.8 is a large effect. Because of the difficulty of interpreting Cohen d from a clinical perspective, we also transformed these values into numbers needed to treat (NNT) according to the formula of Kraemer and Kupfer [44]. The NNT indicates the number of participants who need to be treated to generate one additional clinically significant change [45].

Furthermore, we conducted the following subgroup analyses: the theoretical basis of the intervention (CBT, identified by including cognitive restructuring/challenging dysfunctional thoughts; TWC, identified by inclusion of more recent CBT-based techniques such as mindfulness, meditation, or acceptance of emotions; and alternative interventions [ALT]), guidance (guided with regular written feedback; unguided with no support or only technical support), length of the intervention (short: 1-4 weeks; medium: 5-8 weeks; long: ≥9 weeks), and the risk of bias (low risk=4; some risk<4).

Moreover, a test of homogeneity of the observed effect sizes was calculated using the I² statistic as an indicator of heterogeneity in percentages. Thereby, a value of 0% indicates no heterogeneity, 25% is considered low, 50% is considered moderate, and 75% is considered a high level of heterogeneity [46]. We calculated 95% confidence intervals around I² [47] using the noncentral approach based on chi-square within the heterogi module for Stata [48]. Although we calculated the Q-statistics, we only report whether the result was significant.

Publication bias was investigated by conducting a visual inspection of the funnel plot for the primary outcome measure. An asymmetric as opposed to a symmetric inverted funnel shape indicates potential publication bias that could compromise the conclusions drawn from the meta-analysis. Egger’s test [49] was used to test for the significance of the likely presence of publication bias. Additionally, we performed Duval and Tweedie’s trim-and-fill analysis [50] to verify an unbiased estimate of the pooled effect size. This analysis calculates an estimation of the number of missing studies and the potential effect of these studies on the outcome.

The systematic literature research resulted in 2137 abstracts. An additional nine potentially relevant articles were identified through other searches. After removing the duplicates, we screened the titles and abstracts of 1781 articles and excluded 1687 articles because of their apparent irrelevance. With regard to eligibility, 94 full-text articles were retrieved and assessed by two independent raters (EH and DL); Cohen kappa for agreement between the independent raters was very good (Cohen κ=.83). Any discrepancies were resolved by discussion. We included 27 studies according to the inclusion and exclusion criteria. However, the results of one study [51] were accounted for in one of the other included articles [24] and it was not possible to calculate effect sizes for three studies due to insufficient data [52-54]. In three studies [42,55,56], a small proportion of the participants were younger than 18 years, which was the prespecified criterion of being classified as adults. Because the studies fit all the other inclusion criteria and our sensitivity analyses indicated no difference in the overall results, we decided to include those studies.

Thus, we included 23 studies in the analysis. These 23 studies included 26 comparisons from baseline to posttest. Follow-up data were available for four studies (six comparisons) at 1 to 3 months and for six studies at 4 to 6 months. The PRISMA flowchart of the study selection is presented in Figure 1.

Selected characteristics of the 23 included studies [21-39,42,55-57] are presented in Table 1 (trial characteristics) and Table 2 (intervention characteristics). A more detailed description of the interventions is available in the Multimedia Appendix 2.

The total number of participants was 4226 (intervention groups: n=2312, control groups: n=1914). The included interventions varied according to the intervention content (see Table 2 for details). Most studies evaluated interventions based on TWC (13 comparisons), followed by ALT interventions (7 comparisons), and CBT interventions (6 comparisons). Seventeen studies used a waitlist control comparison, three studies a no-treatment control group, and three studies used an attention control group. The included studies were predominantly conducted in the United States (11 studies), followed by Germany (3 studies), Sweden (2 studies), Austria/Switzerland (2 studies), Japan (2 studies), Norway (1 study), the United Kingdom (1 study), and the Netherlands (1 study).

The interventions of the included studies were mainly Web-based interventions (n=20). For Web-based interventions, an active Internet connection is necessary (eg, to access a website, use a mobile phone app, or visit a virtual classroom). A total of three computer-based interventions [23,25,39] were found in which interventions were installed, for example, on a desktop computer in a separate room at work.

The Perceived Stress Scale (PSS-10, PSS-14) was predominantly used to assess the level of stress (15 of 23 studies). Follow-up assessments have been reported for nine studies (12 comparisons); whereas four studies (6 comparisons) reported data for up to 3 months and six studies (6 comparisons) for 4 to 6 months (see Table 1). Extended follow-ups (no comparison with the respective control groups) were conducted at 1 month (2 studies), 3 months (1 study), 6 months (2 studies), one year (2 studies), and 3 years (1 study). For nine studies, only posttest assessments were available. The participants received guidance in seven studies [24,26,27,34,35,37,42], one study [32] assessed a less intensive guidance format (adherence-focused guidance: combination of reminders and written feedback only on request of the participants), and 16 studies (18 comparisons) investigated unguided interventions. Nine unguided studies reported the use of automated or telephone and mail reminders for completion of the intervention [23,25,28,29,33,36,42,55,56] (Table 2).

A risk of bias was present in all studies. Only six studies fulfilled four of the five criteria used. Nine studies met three criteria, and eight studies fulfilled two criteria. In most cases, the concealment of allocation was insufficiently described, and only 13 of 23 studies reported adequate handling of missing data. In particular, the risk for selective outcome reporting was unclear because the study registration prior to the trials could only be retrieved from three studies [31,32,34]. Although another four studies registered their trial [28,30,33,36], this step occurred retrospectively. The mean interrater reliability between independent raters was κ=.84 and ranged from .60 (selective outcome reporting) to 1.0 (blinding). Disagreements were handled by discussion.

Neither the inspection of the funnel plot nor the Egger’s test [49] indicated a possible publication bias. Moreover, the Duval and Tweedie trim-and-fill analysis [50] indicated no missing studies.

Table 3 presents the effect sizes, confidence intervals, level of significance and heterogeneity for the overall effects on stress, depression, and anxiety as well as for the subgroups. The overall analysis of effect sizes yielded a significant effect size of Cohen d=0.43 for the primary outcome stress across all studies (95% CI 0.31-0.54; n=26). Heterogeneity was moderate (I²=68.01, 95% CI 52.08-78.72). Significant small effect sizes were observed for the secondary outcomes depression (Cohen d=0.34, 95% CI 0.21-0.48; n=13) and anxiety (Cohen d=0.32, 95% CI 0.17-0.47; n=14). Figure 2 displays a forest plot of the effect sizes and the confidence intervals of the included studies.

This meta-analysis has a number of limitations. First, because the risk of bias in the included studies was high, these results must be interpreted with caution. Second, the overall number of studies for the follow-up points and for some subgroups is small, limiting the strength of conclusions that can be drawn from these results. Third, we found a moderate degree of heterogeneity for the primary outcome that was reduced when analyzing the level of risk of bias and guidance subgroups; nevertheless, the number of comparisons in some subgroups was small and did occasionally overlap concerning individual studies. Fourth, we are mindful of the possibility that despite our thorough literature search, we might have missed a relevant study. Finally, because of the lack and inconsistency of information provided by the included studies, we were unable to analyze the effect of potentially relevant moderators of the treatment effect, such as the effect of adherence to the intervention on the overall outcome.

In future studies, preregistration of trials, an adequate calculation of the sample size, a more detailed description of allocation concealment, and appropriate methods to account for missing values are strongly recommended. We observed that a growing number of studies adhered to the intention-to-treat principle (ie, by employing mixed-effects models); nevertheless, especially for those studies, we recommend to ensure that the reported descriptive statistics are based on adequate methods to handle missing values and do not solely rely on complete cases. This will ensure adherence to the intention-to-treat principle throughout all statistics.

Although in most studies the therapeutic approach that was used in the intervention (eg, CBT) is often well described, it would be desirable for future studies to also specify the theoretical model that was used to develop the intervention (eg, conservation of resources theory, transactional model of stress).

Overall, more methodologically rigorous studies with a low risk of bias are needed to assess the effect of, for example, particular characteristics of interventions, such as treatment latitude, different levels of guidance, and different types of interventions in a direct comparison. Moreover, information on the number of participants who adhered to the intervention and details on co-interventions alongside training would be insightful. Future research should also test Web- and computer-based stress management interventions against the highest standard in this field (ie, classical face-to-face stress management interventions) and should more frequently include longer follow-up periods (eg, up to 6 months). Data on the cost-effectiveness of such interventions would also be highly relevant.

Despite the limitations discussed, it appears safe to conclude that Web- and computer-based interventions can be effective. In particular, interventions that include guidance from an online coach, are of medium length, or that are based on (third-wave) CBT lead to moderate improvements in stress levels. Initial evidence also suggests that the effects can be maintained up to 6 months. Whereas research and practice on traditional face-to-face interventions have been prolific [8,9], research on the efficacy and dissemination of Web-based stress-management interventions is still at the beginning despite the high potential and reach of such interventions. This work draws attention to the need for further studies on the efficacy, cost-effectiveness, and mechanisms of change of such interventions. In summary, the integration of evidence-based Web-based stress management interventions into health care systems has the potential to make a valuable contribution to reducing stress-related mental health problems on a large scale.