Background: Mental health conditions are considered the leading cause of disability, sickness absence, and long-term work incapacity. eHealth interventions provide employees with access to psychological assistance. There has been widespread implementation and provision of eHealth interventions in the workplace as an inexpensive and anonymous way of addressing common mental disorders.
Objective: This updated review aimed to synthesize the literature on the efficacy of eHealth interventions for anxiety, depression, and stress outcomes in employee samples in organizational settings and evaluate whether their effectiveness has improved over time.
Methods: Systematic searches of relevant articles published from 2004 to July 2020 of eHealth intervention trials (app- or web-based) focusing on the mental health of employees were conducted. The quality and bias of all studies were assessed. We extracted means and SDs from publications by comparing the differences in effect sizes (Hedge g) in standardized mental health outcomes. We meta-analyzed these data using a random-effects model.
Results: We identified a tripling of the body of evidence, with 75 trials available for meta-analysis from a combined sample of 14,747 articles. eHealth interventions showed small positive effects for anxiety (Hedges g=0.26, 95% CI 0.13-0.39; P<.001), depression (Hedges g=0.26, 95% CI 0.19-0.34; P<.001), and stress (Hedges g=0.25, 95% CI 0.17-0.34; P<.001) in employees’ after intervention, with similar effects seen at the medium-term follow-up. However, there was evidence of no increase in the effectiveness of these interventions over the past decade.
Conclusions: This review and meta-analysis confirmed that eHealth interventions have a small positive impact on reducing mental health symptoms in employees. Disappointingly, we found no evidence that, despite the advances in technology and the enormous resources in time, research, and finance devoted to this area for over a decade, better interventions are being produced. Hopefully, these small effect sizes do not represent optimum outcomes in organizational settings.
Trial Registration: PROSPERO CRD42020185859; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=185859
Mental health conditions are to be considered the leading cause of disability, sick leave, and long-term work incapacity in most developed countries . Furthermore, poor mental health has a substantial impact on employee well-being, productivity, absenteeism, compensation claims, and social welfare systems [ , ]. Evidence supports the increased demand for workplace interventions, highlighting that working conditions and the workplace environment can influence employees’ mental health and well-being [ ].
The nature of mental health symptoms fluctuates on a continuum between thriving and struggling . Most mental ill health that is seen in the workforce is because of common mental disorders, most notably, depression, anxiety, and stress-related conditions [ , , ]. Employees who have a mental health condition and become too unwell to continue working rarely move straight from being healthy to needing sick leave [ ]. There is usually a course that an employee might experience as they develop worsening symptoms [ ], and different interventions may be required at different stages of this course.
International approaches suggest multilevel organizational approaches targeting (1) healthy workers via universal prevention interventions; (2) those with subclinical conditions (symptomatic or at-risk workers), such as those experiencing high stress via indicated interventions; and (3) those workers who have disclosed a mental health condition with tertiary interventions (treatments) . Furthermore, the UK Thriving at Work Review into Mental Health at Work recommends the inclusion of the use of therapeutically tailored interventions based on individual-specific needs [ ].
The 2 potential rate-limiting steps for organizations in implementing mental health intervention programs are the budget and logistics of delivering universal interventions to all staff at scale and the ability to target indicated and early interventions to those who are at risk or unwell before disclosure. The internet offers a unique opportunity to address these rate-limiting steps by delivering eHealth interventions with components such as cognitive behavioral therapy (CBT) and stress management to a broad audience. eHealth interventions provide employees with access to psychological help when they are not employed in a typical working environment, such as shift workers or those working from home, or when they may be sick-listed from work. As a result, eHealth interventions have been widely implemented and provided. eHealth is an emerging field in public health and business and provides health services and information delivered or enhanced through the internet and related technologies, such as smartphone apps. Recently, eHealth has been seen as a popular approach in organizations as it provides an inexpensive and anonymous way of addressing common mental disorders , including apps linked to wearable devices, and guided meditation programs [ ].
There is a plethora of evidence for the short- and long-term benefits of eHealth-delivered CBT for treating anxiety and depressive conditions in both the general population and clinical settings [- ]. Evidence has also emerged for the effectiveness of mindfulness-based eHealth interventions in improving symptoms for both the general population and individuals who are symptomatic [ ].
However, we know that employed individuals differ systemically from both general and clinical populations used in most eHealth studies; for instance, employees have much better mental health (fewer symptoms) than general and clinical populations , for whom many eHealth interventions and their content have been developed. As such, there are likely to be floor effects and other efficacy modifiers. The delivery in, and by, organizations will be different from just open access to interventions in the general population. This influences uptake and engagement, which are known determinants of digital health efficacy [ ].
A previous meta-analysis considering eHealth interventions from 2004 (first identified) to 2017  found randomized controlled trial (RCT) evidence for only 23 eHealth interventions delivered to employees. Overall, there was a small pooled effect of reducing depression, anxiety, and stress symptoms after the intervention (Hedges g=0.24), which was sustained at follow-up (Hedges g=0.23). Similar effect sizes (ESs) were found in a contemporaneous review [ ]. Only 2 years later, a meta-analysis [ ] identified 50% (n=34) more studies of such interventions and suggested stronger effects on stress (Hedges g=0.54), insomnia (Hedges g=0.70), and burnout (Hedges g=0.51) but not symptoms of depression or anxiety.
None of the previous reviews have addressed the key issue of efficacy, which is affected by the digital placebo effect. Smartphone app users can experience significantly reduced mental health symptoms, even if the app does not contain any direct therapeutic intervention, similar to the placebo effect seen in pharmaceutical trials when using “active” or “attention” controls , or may reflect regression to the mean of fluctuating symptoms. Even simply tracking the symptoms of depression in an app can lead to significant reductions in depression [ ]. Both processes result in an apparent “intra-group” effect without any therapeutic intervention. In a trial, the type of control used is a primary determinant of the “between-group” ES. In a recent systematic review of smartphone apps for anxiety in clinical samples, the between-group ESs were lower in trials that used active controls than in those that used passive controls [ ]. This inflates the apparent efficacy of the interventions evaluated in trials with passive controls, such as the wait-list. It is not known whether and to what extent this is observed in working populations or for more preventive approaches.
Considering the increasing number of interventions being developed, implemented, and potentially evaluated, this review aimed to address the following questions by systematically reviewing the current state of evidence for the efficacy of eHealth interventions in reducing depression, anxiety, and stress in employees ().
Research questions addressed by this review
- Is there evidence for the improvement in effectiveness over time (ie, are the interventions getting better)?
- What factors, if any, moderate the efficacy?
- Intervention approach; indicated, tailored, tertiary or universal
- Type of control (active vs wait-list)
- Type of intervention: cognitive behavioral therapy, mindfulness, stress management, and other
- Presence of in-person support
This review complies with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines . We aimed to identify all published and unpublished, peer-reviewed, controlled clinical trials of eHealth interventions targeted at employees reporting outcomes on a standardized mental health measure of depression, anxiety, and stress. The systematic review protocol was registered on PROSPERO (International Prospective Register of Systematic Reviews; CRD42020185859).
A systematic literature search was conducted using MEDLINE, PsycINFO, Cochrane Register of Controlled Trials, and Embase electronic databases for relevant peer-reviewed articles in English of controlled trials and RCTs published from 2004 (when the first eHealth intervention was identified)  to July 2020. Keywords used related to “workplace,” “intervention,” “outcome,” and “study design.” An example of this search strategy is presented in . The search terms were developed from our previous systematic review [ ]. The tables of contents of the Journal of Medical Internet Research, Journal of Internet Interventions, Occupational and Environmental Medicine, and the Journal of Occupational and Environmental Medicine, as well as the reference lists of included studies, were manually searched.
Study Selection Criteria
The PICOS (Patient/Population, Intervention, Comparison, and Outcomes) framework commonly used to identify components of clinical evidence for systematic reviews in evidence-based medicine, endorsed by the Cochrane Collaboration , was used in this review.
Participants had to be in current paid employment and working-age adults aged between 18 and 65 years. Studies were excluded if the sample was defined as volunteer workers, unemployed participants, or general or clinical populations.
Any eHealth intervention, defined as a therapeutic intervention delivered through a website, smartphone, tablet, or mobile app and designed to improve mental health, was included.
Studies were required to have a control group, defined as either passive (care as usual, no contact, or wait-list) or active (eg, another eHealth intervention as the comparison group).
The study had to report on at least one common mental disorder outcome: depression, anxiety, or stress. All eligible outcomes for each study and domain were included. The outcomes were standardized mean difference (SMD) from preintervention baseline score to immediate postintervention use and the follow-up time point.
The following measurements of common mental disorders were regarded as eligible:
- Diagnostic interview
- Self-report diagnosis by a physician, psychologist, or other qualified health professional
- Self-administered rating scale for mental health (anxiety, depression, or stress)
Studies were excluded from the review if they were not specifically limited to employees and delivered in a workplace setting or used in-person, telephone, and email interventions only.
Identification of Studies
After duplicates were removed, 3 independent authors (ES, IC, and MA) screened all titles and abstracts to identify potentially relevant studies. Abstracts and full-text versions of potentially eligible studies were independently assessed by 2 investigators (ES and IC). Eligible studies with individual citations were scanned to ensure that all relevant studies were identified. Disagreements were adjudicated in conjunction with the senior author (NG).
Data Extraction and Coding
We extracted the mean and SD of standardized measures of stress or distress, depression, and anxiety and the sample size (n) in each arm (intervention and control) at baseline and at each follow-up time point. When sufficient details were not reported, the authors were contacted. Additional data concerning the type of design, intervention and control details, participant characteristics, study duration, length of follow-up, organizational population, and general outcomes were recorded (see the Results section). Studies were also coded by intervention type: universal interventions targeting relatively healthy participants; indicated interventions targeting subclinical symptoms or syndromes; and tertiary treatments for explicit diagnoses.
Studies reporting results of the same intervention and sample in different papers (eg, a postintervention and follow-up study) were treated as 1 study, and we used the first follow-up outcome point in the analysis.
We categorized studies according to their mental health outcome measures: anxiety, depression, or stress. Studies with multiple outcomes appear in >1 mental health outcome result.
Quality Assessment and Risk of Bias Within Studies
The risk of bias in RCTs was assessed using the revised Cochrane risk of bias tool for RCTs (RoB version 2.0) , in which 5 domains were independently evaluated by 2 authors: randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result. Each domain was assessed for the risk of bias. Studies were graded as (1) “low risk of bias” when a low risk of bias was determined for all domains, (2) “some concerns” if at least one domain was assessed as raising some concerns but not to be at high risk of bias for any single domain, or (3) “high risk of bias” when a high risk of bias was determined for at least one domain or there were bias concerns in multiple domains [ ]. If a study contained >1 intervention arm, each intervention was assessed as 1 study.
The summary ES was the SMD (calculated as Hedge g, with 95% CI) between the intervention and control groups for each outcome measure (anxiety, depression, and stress) . A positive ES (SMD) indicated that the intervention was more efficacious than the control. The level of significance was set at P<.05 and 95% CIs. The magnitude of the effect was categorized as large (SMD>0.8), moderate (SMD 0.5-0.8), small (SMD 0.2 to <0.5), or trivial (SMD<0.2) [ ]. Pooling of ESs across studies was performed using a random-effects model in Comprehensive Meta-Analysis (Biostat Inc; version 3). We detected and accommodated outliers in the meta-analysis using the random-effects variance shift model and the likelihood ratio test. As the different types of eHealth interventions and approaches used may result in large heterogeneity [ ], between-study heterogeneity was quantified using τ2 (variance of true effects) and further assessed using the Higgins I2 statistic, which estimates the proportion of observed variance not because of sampling error. An I2 value of 75% was considered large, 50% was considered moderate, and 25% was considered low [ ].
We estimated the results separately for each mental health outcome measure (anxiety, depression, and stress) at the postintervention and first follow-up time points. Meta-regressions were performed for each mental health outcome to identify whether the intervention approach (eg, universal), type of intervention (eg, CBT), type of control (eg, wait-list), or intervention delivery method (web-based vs smartphone) moderated the observed effect.
To evaluate differences in effectiveness over time, we used only within-group ESs from the intervention arms for anxiety, depression, or stress outcomes. ESs were calculated by dividing the difference between pre- and posttreatment means by the pooled SD of the difference . ANOVA was used to assess whether there were any differences between the mean ESs for each year. For a test for trends, the within‐group ESs with positive signs indicated improvement, and negative signs indicated worsening. This method allowed for the difference in effects over time to be accurately calculated, allowing us to explore whether interventions improved with technology.
To assess small study effects (in cases where at least 10 studies were available for analysis), we used a funnel plot for the overall effects and each subgroup analysis, which compared the outcome effects with their SEs. We used the Egger regression test to examine further asymmetry of the funnel plot  with statistical significance based on P<.10. In cases where at least 10 studies were available and a small study effect was found, we used a Duval and Tweedie trim and fill analysis to quantify the magnitude of the small study effect [ ].
The search strategy identified 3411 titles (). Of the 3411 articles, after the removal of 371 (10.88%) duplicates, 3040 (89.12%) titles and abstracts were reviewed by the authors (ES, IC, and MA). Of these 3040 articles, 2893 (95.16%) articles were excluded based on the eligibility criteria, leaving 147 (4.84%) articles that were potentially relevant to the research question. Their full texts were examined by 2 independent researchers (ES and MA), and discrepancies were decided by the senior author (NG). Of the 147 articles, 74 (50.3%) were excluded for reasons provided in . Of the remaining 73 studies, data were missing for 8 (11%) studies. The authors were contacted, and 5 responded with relevant data [ - ]. A total of 3 authors did not respond or were unable to obtain data; thus, these studies were excluded from the meta-analysis [ , , ]. Of the 73 studies, 4 (5%) articles reported on the same study at different follow-up time points and were merged [ - ], leaving 68 (93%) studies, of which 2 (3%) were excluded as they used face-to-face treatment for control groups [ , ]. The remaining 97% (66/68) of articles were identified as meeting the criteria for quality assessment. Of the 66 studies, 5 (8%) used 2 intervention arms [ - ], and 2 (3%) studies used 3 intervention arms [ , ]. In this case, the intervention arms were treated as individual trials reported as author names; year of publication; and the letters a, b, or c (eg, Smith [a]). The number of participants in the control group was split evenly as a comparative arm to ensure that participants were not counted twice. This process resulted in 75 trials ( ) for the meta-analysis. [ - , - , - ] provides the list of references.
Risk of Bias
The individual studies included in the meta-analysis showed that, overall, 12% (9/75) had a low risk of bias, 3% (2/75) had ≥1 concern, and 85% (64/75) had a high risk of bias. Most of the bias was observed in deviations from the intended interventions and missing outcome data, which is not uncommon and is an area for improvement in the eHealth field. An important aspect common to behavioral interventions is that 55% (41/75) of the studies were rated as having a high risk of bias because of the use of a wait-list control group. As such, it was impossible for users to remain blinded ([ - , - , - ]).
All the included trials used web-based self-report measures to collect the baseline and follow-up data.provides an overview of the scales used and their prevalence.
Overall Summary of the Identified Interventions
The 75 trials reported postintervention data for 14,747 participants. Most trials used universal approaches (47/75, 63%), with fewer indicated (10/75, 13%), tertiary (11/75, 15%), or tailored (7/75, 9%) intervention approaches. Most interventions were delivered via web-based platforms as opposed to smartphone apps. Most of the trials used a wait-list control group. The 3 most common types of interventions described by the authors were CBT, mindfulness, and stress management. The studies were conducted in 15 countries: the United States (16/75, 21%), Japan (15/75, 20%), Germany (13/75, 17%), the United Kingdom (7/75, 9%), the Netherlands (5/75, 7%), Australia (4/75, 5%), Sweden (4/75, 5%), China (2/75, 3%), Europe combined (2/75, 3%), Italy (2/75, 3%), Brazil (1/75, 1%), Finland (1/75, 1%), Hong Kong (1/75, 1%), Singapore (1/75, 1%), and the United States and Canada combined (1/75, 1%). The most common types of participants were health care professionals (18/75, 24%). Interventions for the insurance industry (7/75, 9%), managers (6/75, 8%), information technology (6/75, 8%), male-dominated industries (5/75, 7%), telecommunications (5/75, 7%), schools (3/75, 4%), universities (3/75, 4%), marketing and sales (3/75, 4%), banking (1/75, 1%), and human resources (1/75, 1%) were also evaluated (). The term general employee is used in when the study did not mention the organization type. A total of 15 trials were conducted across several industries, and only 2 were conducted in employees on sick leave. Further summarized and detailed descriptions of the studies are presented in and [ - , - , - ].
|Study||Type||Country and population (sample size)||Intervention and duration (sample size)||Control (sample size)||Type of intervention|
|Grimea ||CBTb||United Kingdom; Occupational health department (48)||WLCc (24)||Tertiary|
|Hasson et ala,d ||CBT||Sweden; IT and media companies (317)||WLC+information (174)||Universal|
|Shimazu et ala ||CBT||Japan; construction machinery company (225)||WLC (113)||Universal|
|Cook et ala ||Stress management||United States; human resource employees (419)||WLC+paper-based information(210)||Universal|
|Ruwaard et ala,d ||CBT||The Netherlands; general employees (239)||WLC (62)||Tailored|
|Billings et ale ||CBT||United States; technology company (309)||WLC (155)||Tailored|
|Suzuki et ala ||CBT||Japan; university staff (43)||WLC (22)||Universal|
|Yamagishi et ala ||Assertion training||Japan; shift working nurses (60)||WLC (30)||Universal|
|Abbott et ala ||CBT||Australia; industrial organization (53)||WLC (27)||Universal|
|Bennett et ala ||Behavior change||United States; general managers (145)||WLC (73)||Universal|
|Glück and Maerckera ||Mindfulness||Austria, Germany, and Switzerland; universities, car dealerships, broadcasting stations, and health care consulting (50)||WLC (21)||Universal|
|Borness et ala ||Cognitive training||Australia; public sector general employees (135)||Active control program (68); general knowledge information||Universal|
|Feicht et ala ||Positive psychology||Germany; insurance company (147)||WLC (62)||Universal|
|Ketelaar et al a||Health Surveillance Model||The Netherlands; nurses and allied health professionals (367)||WLC (188)||Tailored|
|Lappalainen et ala,d ||CBT and ACTf||Finland; men aged 28 to 58 years with depression (23)||WLC (12)||Tertiary|
|Villani et ala,d ||Stress management||Italy; female oncology nurses with high stress (30)||Active control with 8 video clips (15)||Indicated|
|Bolier et ala,d ||Health Surveillance Model||The Netherlands; nurses, allied health professionals, and general employees (423)||WLC (211)||Tailored|
|Deitz et ala ||Behavior change||United States; hospital employees with cardiovascular risk (210)||WLC (105)||Universal|
|Ebert et ala,d ||Problem-solving training||Germany; teachers (150)||WLC (75)||Tertiary|
|Geraedts et al [, ] (2 papers)a,d||Problem-solving training and cognitive therapy||The Netherlands; banking (231)||WLC (115)||Tertiary|
|Imamura et al [, ] (2 papers)a,d||CBT||Japan; IT (762)||WLC+information (381)||Tertiary|
|Ly et ala,d ||ACT||Sweden; middle managers in the private sector (73)||WLC (37)||Universal|
|Mori et ala,d ||CBT||Japan; IT engineers with high computer literacy (168)||WLC (83)||Universal|
|Phillips et ala ||CBT||United Kingdom; transport, health, and communication sectors (637)||WLC+information (319)||Tertiary|
|Umanodan et ala ||Stress management||Japan; manufacturing company (266)||WLC (121)||Universal|
|Carissoli et ale ||Mindfulness||Italy; general employees (56)||Music; 2 songs per day, lasting approximately 15 minutes each, while doing nothing else (18)||Universal|
|Cook et ala ||Stress management||United States; older IT employees (278)||WLC (140)||Universal|
|Ebert et ala ||Behavior change||Germany; teachers (128)||WLC (64)||Universal|
|Guille et al a||CBT||United States; medical interns (199)||4 weekly mental health information emails (99)||Universal|
|Mak et al a||MBSRg||China; university (321)||WLC (107)||Universal|
|Praseka ||Mindfulness||Unite States; university (192)||WLC (91)||Universal|
|Stansfeld et ala,d ||Stress management||United Kingdom; NHSh Mental Health Trust (275)||WLC (59)||Universal|
|Volker et al a||CBT and PSTi||The Netherlands; sick-listed (773)||WLC (89)||Tertiary|
|Yuana ||PST||Hong Kong||WLC (159)||Universal|
|Allexandre et al a||Mindfulness||United States; general employees from a corporate call center (91)||WLC (37)||Universal|
|Birney et ala ||CBT||United States (300)||WLC (150)||Tertiary|
|Bostock et ala ||CBT||United States; office based (270)||WLC (135)||Universal|
|Dyrbye et ala ||Positive psychology||United States; practicing physicians (290)||WLC (145)||Universal|
|Ebert et ala,d  (a)||Stress management||Germany (264)||WLC (132)||Indicated|
|Ebert et ala,d  (b)||Stress management||Germany (264)||WLC (132)||Indicated|
|Heber et ala,d ||Stress management||Germany (264)||WLC (132)||Indicated|
|Hersch et ala ||Stress management||United States; nurses and nurse managers in public hospitals (104)||WLC (52)||Universal|
|Imamura et ala ||CBT||Japan (1236)||WLC no depression (285); same as above but for moderate depression (290); same as above but for high depression (43)||Tailored|
|Jonas et ala,d ||CBT||Germany; employees with burnout (59)||WLC (21)||Indicated|
|Beiwinkel et ala,d ||CBT||Germany (180)||WLC+information (80)||Tertiary|
|Boß et al  (a)a and (b)a,d||CBT||Germany (434)||WLC (144)||Universal|
|Carolan et al a,d||CBT||United Kingdom (84)||WLC (28)||Indicated|
|Shirotsuki et al a||CBT||Japan; office workers in hospitality and sales company (87)||Recorded their mood state every day on a weekly monitoring sheet (homework; 29)||Universal|
|Zhang et ala ||Awareness training||Singapore; health care workers (80)||Information from the Health Promotion Board (40)||Universal|
|Zwerenz et ala,d ||Psychoeducation||Germany; sick-listed and in inpatient rehabilitation (652)||Active control; regular email reminders to use selected information posted on the web about stress management and coping (329)||Tertiary|
|Bostock et ala ||Mindfulness||United Kingdom; pharmaceutical and high-tech employees (238)||NHS web-based advice for work stress (110)||Universal|
|Eriksson et ala ||Mindful self-compassion program||Sweden; practicing psychologists (101)||WLC (49)||Universal|
|Gollwitzer et al a||Mental contrasting||Germany; general nurses (129)||WLC (47)||Universal|
|Hamamura et ale ||CBT||Japan; marketing company (557)||WLC (251)||Indicated|
|Imamura et al a,d||CBT||Japan; telecommunications (706)||WLC (353)||Universal|
|Lilly et ala ||MBSR||United States and Canada; emergency telecommunication (323)||WLC (160)||Universal|
|Mistretta et ala ||MBRTj||United States; hospital employees (38)||WLC (15)||Indicated|
|Oishi et ala ||CBT||Japan; teachers (240)||WLC (120)||Universal|
|Persson Asplund et ala,d ||Stress management||Sweden; middle managers in health care, education, IT, or communications sectors (117)||Active control; weekly mail contact, homework, and access to a moderated discussion forum with other users (58)||Indicated|
|Querstret et ala ||MBCTk||United Kingdom (118)||WLC (58)||Universal|
|Song et ala ||Health Surveillance Model||Japan (1526)||WLC (914)||Universal|
|Coelhoso et ala,d ||Mindfulness meditation||Brazil; female private hospital employees (490)||Active control; similar program (240)||Universal|
|Stratton et ala ||Disclosure decision aid tool||Australia (107)||Information provided about disclosure on leading NGOl website (54)||Tertiary|
|Weber et ala ||Behavior change||Germany, England, and Northern Ireland (532)||WLC (322)||Universal|
|Deady et ala ||Behavior activation and mindfulness||Australia; male-dominated industries (2257)||Active control same intervention but without risk calculator and mood tracker (1144)||Universal|
aRandomized controlled trial.
bCBT: cognitive behavioral therapy.
cWLC: wait-list control.
fACT: acceptance and commitment therapy.
gMBSR: mindfulness-based stress reduction.
hNHS: National Health Service.
iPST: problem-solving therapy.
jMBRT: mindfulness-based resilience training.
kMBCT: mindfulness-based cognitive therapy.
lNGO: nongovernmental organization.
Efficacy for Anxiety
Overall, the 29 trials that measured anxiety outcomes showed a significant, small positive effect at the postintervention time point (Hedges g=0.26, 95% CI 0.13-0.39; P<.001), with high heterogeneity (I2=77.49%; τ2=0.08). Approximately 21% (16/75) of studies reported follow-up outcomes, showing a small positive effect (Hedges g=0.32, 95% CI 0.15-0.50; P<.001), with similarly large heterogeneity (I2=76.55%; τ2=0.08;).
Efficacy for Depression
An overall small positive effect on depression at the postintervention time point (Hedges g=0.26, 95% CI 0.19-0.34; P<.001) was estimated by combining 46 trials. Moderate heterogeneity was observed (I2=66.96%; τ2=0.04). At the postintervention time point, one outlier was detected. By removing this outlier with a very large ES, a statistically significant but small magnitude positive effect remained (Hedges g=0.24, 95% CI 0.17-0.32; P<.001). Approximately 60% (45/75) of the studies reported follow-up outcomes. The follow-up effects on depression in 37% (28/75) of studies were very similar (Hedges g=0.23, 95% CI 0.13-0.32; P<.001), again with moderate heterogeneity (I2=68.27%; τ2=0.03;).
Efficacy for Stress
Stress was the most common mental health outcome assessed in this study. In 76% (57/75) of studies, a small positive effect was found overall at the postintervention time point (Hedges g=0.25, 95% CI 0.17-0.34; P<.001). However, large heterogeneity was detected (I2=76.11%; τ2=0.07). Overall, a significant but small positive effect was observed at follow-up in 40% (30/75) of studies (Hedges g=0.28, 95% CI 0.17-0.40; P<.001), with moderate heterogeneity detected (I2=73.36%; τ2=0.06;).
Moderating Factors for Outcome Efficacy
Mixed-effects meta-regression () showed that for anxiety, the difference between approach efficacy was statistically significant (Q=21.72; P<.001; R2=0.64). Stress management interventions (Hedges g=0.79, 95% CI 0.64-0.93; P<.001) and mindfulness (Hedges g=0.42, 95% CI 0.14-0.60; P<.001) interventions were more effective than CBT (Hedges g=0.11, 95% CI 0.04-0.19; P=.004) and other interventions (eg, cognitive training; Hedges g=0.13, 95% CI 0.01-0.25; P=.04). There were no significant between-group differences between the intervention approach, level of support, or type of control used.
|Intervention component and study design factor||Study, n (%)||Meta-analysis||Heterogeneity||Meta-regression between-group tests|
|Hedges g (95% CI)||P value||Q||P value||I2||Q||P value||R2|
|Type of intervention|
|CBTa||14 (19)||0.11 (0.04 to 0.19)||.004||14.41||.35||9.78||21.72||<.001||0.64|
|Mindfulness||5 (7)||0.42 (0.14 to 0.6)||<.001||27.99||<.001||85.71||21.72||<.001||0.64|
|Stress management||4 (5)||0.79 (0.64 to 0.93)||<.001||6.47||.09||53.6||21.72||<.001||0.64|
|Other||6 (8)||0.13 (0.01 to 0.25)||.04||5.45||.35||8.2||21.72||<.001||0.64|
|Indicated||9 (12)||0.44 (0.33 to 0.54)||<.001||60.13||<.001||86.70||3.48||.32||0.03|
|Tailored||3 (4)||0.06 (−0.09 to 0.22)||.44||0.76||.68||0.000||3.48||.32||0.03|
|Tertiary||5 (7)||0.14 (0.04 to 0.23)||.006||2.85||.58||0.000||3.48||.32||0.03|
|Universal||12 (16)||0.29 (0.17 to 0.4)||<.001||38.03||<.001||71.07||3.48||.32||0.03|
|Yes||14 (19)||0.32 (0.24 to 0.4)||<.001||68.96||<.001||81.15||1.02||.31||0.01|
|No||15 (20)||0.16 (0.07 to 0.24)||<.001||47.33||<.001||70.42||1.02||.31||0.01|
|Type of control|
|Active||4 (5)||0.26 (0.07 to 0.45)||.009||5.14||.16||41.64||0.08||.78||0|
|WLCb||25 (33)||0.25 (0.19 to 0.31)||<.001||119.23||<.001||79.87||0.08||.78||0|
|Type of intervention|
|CBT||23 (31)||0.11 (0.06 to 0.17)||<.001||44.12||.003||50.13||20.32||<.001||0.47|
|Mindfulness||5 (7)||0.46 (0.28 to 0.64)||<.001||16.14||.003||75.22||20.32||<.001||0.47|
|Stress management||4 (5)||0.61 (0.47 to 0.75)||<.001||4.03||.26||25.52||20.32||<.001||0.47|
|Other||14 (19)||0.15 (0.09 to 0.21)||<.001||21.25||.07||38.83||20.32||<.001||0.47|
|Indicated||9 (12)||0.32 (0.21 to 0.42)||<.001||45.95||<.001||85.59||4.45||.22||0.01|
|Tailored||6 (8)||0.04 (−0.06 to 0.14)||.41||12.13||.03||58.79||4.45||.22||0.01|
|Tertiary||11 (15)||0.2 (0.12 to 0.28)||<.001||18.98||.04||47.31||4.45||.22||0.01|
|Universal||20 (27)||0.19 (0.13 to 0.25)||<.001||44.01||.001||56.83||4.45||.22||0.01|
|Yes||20 (27)||0.33 (0.27 to 0.4)||<.001||42.04||.002||54.8||7.20||.007||0.27|
|No||26 (35)||0.11 (0.06 to 0.16)||<.001||65.58||<.001||61.88||7.20||.007||0.27|
|Type of control|
|Active||9 (12)||0.19 (0.11 to 0.28)||<.001||16.78||.03||52.33||0.00||.95||0.00|
|WLC||37 (49)||0.18 (0.14 to 0.23)||<.001||119.35||<.001||69.84||0.00||.95||0.00|
|Type of intervention|
|CBT||19 (25)||0.1 (0.04 to 0.16)||.001||47.89||<.001||62.42||10.12||.02||0.09|
|Mindfulness||15 (20)||0.42 (0.31 to 0.53)||<.001||46.31||<.001||69.77||10.12||.02||0.09|
|Stress management||14 (19)||0.28 (0.2 to 0.37)||.006||71.38||<.001||88.79||10.12||.02||0.09|
|Other||9 (12)||0.12 (0.04 to 0.21)||<.001||37.74||<.001||65.56||10.12||.02||0.09|
|Indicated||9 (12)||0.38 (0.28 to 0.49)||<.001||56.76||<.001||85.91||3.26||.35||0.00|
|Tailored||6 (8)||0.07 (−0.03 to 0.16)||.16||27.10||<.001||81.55||3.26||.35||0.00|
|Tertiary||5 (7)||0.12 (0.09 to 0.22)||.01||4.78||.31||16.3||3.26||.35||0.00|
|Universal||37 (49)||0.18 (0.13 to 0.24)||<.001||125.06||<.001||71.21||3.26||.35||0.00|
|Yes||22 (29)||0.33 (0.26 to 0.39)||<.001||80.20||<.001||73.82||6.79||.009||0.15|
|No||35 (47)||0.09 (0.04 to 0.14)||.001||121.76||<.001||72.08||6.79||.009||0.15|
|Type of control|
|Active||7 (9)||0.17 (0.05 to 0.29)||.006||18.38||.005||67.36||0.03||.87||0.00|
|WLC||50 (67)||0.18 (0.14 to 0.23)||<.001||215.97||<.001||77.31||0.03||.87||0.00|
aCBT: cognitive behavioral therapy.
bWLC: wait-list control.
Similarly, significant differences were observed in the ESs for depression across the types of interventions (Q=21.72; P<.001; R2=0.64). Again, stress management interventions (Hedges g=0.61, 95% CI 0.47-0.75; P<.001) and mindfulness (Hedges g=0.46, 95% CI 0.28-0.64; P<.001) interventions were more effective than CBT (Hedges g=0.11, 95% CI 0.06-0.17; P<.001) and other interventions (Hedges g=0.15, 95% CI 0.09-0.21; P<.001). Supported interventions had a higher ES (g=0.33, 95% CI 0.27-0.40; P<.001) than unsupported (Hedges g=0.11, 95% CI 0.06-0.16; P<.001) interventions (difference Q=7.20; P=.007; R2=0.27). There were no significant differences between the intervention approach and the type of control used.
Differences were observed in the ESs across the type of intervention (Q=10.12; P=.02; R2=0.09). Mindfulness interventions had the largest ES (Hedges g=0.42, 95% CI 0.31-0.53; P<.001), followed by stress management (Hedges g=0.28, 95% CI 0.20-0.37; P=.006), which was more effective than CBT (Hedges g=0.10, 95% CI 0.04-0.16; P=.001) and other interventions (Hedges g=0.12, 95% CI 0.04-0.21; P<.001). Supported interventions had a higher ES (Hedges g=0.33, 95% CI 0.26-0.39; P<.001) than unsupported (Hedges g=0.09, 95% CI 0.04-0.14; P=.001). The difference between these factors was statistically significant (Q=6.79; P=.009; R2=0.15). For stress, there were no significant differences between the intervention approach and type of control used.
Effectiveness of eHealth Interventions Over Time
Over 5 times more studies evaluating eHealth interventions have been published since 2013 (n=64) than between 2004 and 2013 (n=11). However, the effectiveness of eHealth interventions did not seem to improve over time. The mean within-group ES reported in each year of the eHealth intervention arms in trials published since 2004 remained unchanged, with no significant SMD observed over time for anxiety (F1,9=0.28; P=.61), depression (F1,10=0.31; P=.59), or stress (F1,11=0.75l; P=.41;). There appears to have been a nadir in the effectiveness of studies published immediately after the financial crash and recession of 2008/2009.
Small Study Effect
A funnel plot for each mental health outcome is shown in. No significant asymmetry was found for anxiety outcomes (n=29; Egger intercept 0.24; P=.80). However, significant asymmetry was observed for depression (n=46; Egger intercept 1.71; P=.001). After conducting a trim and fill analysis, 3% (2/75) of studies were imputed; the observed postintervention ES was adjusted to Hedges g=0.28 (95% CI 0.18-0.39). Similarly, significant asymmetry was observed in stress (n=57; Egger intercept 1.89; P=.004). After conducting a trim and fill analysis, 23% (17/75) of studies were imputed; the observed postintervention ES was adjusted to Hedges g=0.08 (95% CI 0.00-0.19), suggesting a greater effect reported in smaller studies.
This updated review aimed to synthesize the burgeoning literature on the efficacy of eHealth interventions for anxiety, depression, and stress outcomes in organizational settings and employee samples. We identified 52 new trial interventions published since the 23 identified in our prior review in 2017 , a tripling of the body of evidence. The systematic search identified 75 relevant trials for the meta-analysis, delivering eHealth interventions either on the web or via a smartphone, with a combined sample of 14,747 employees.
eHealth interventions reduced mental health symptoms immediately after use, with small positive effects observed in anxiety (Hedges g=0.26), depression (Hedges g=0.26), and stress (Hedges g=0.25), and data from trials with longer follow-up periods showed similar effects. These results are comparable with those of previous reviews, where small overall effects were found at the postintervention and follow-up time points [, ]. These results imply that since 2017, the efficacy of eHealth interventions compared with control conditions reported in trials has remained unchanged, suggesting that the effectiveness of eHealth interventions does not seem to be improving over time. An analysis of the within-group effect observed in the intervention arms confirmed that since 2004, there has been no apparent systematic improvement in the effectiveness of these interventions. This is a surprising finding, given the enormous literature on methods to improve engagement with such interventions, greater penetration of technology to populations that do not access health care, and the increasing number of interventions delivered as apps, which reportedly improve access. All of these are commonly cited as factors favoring eHealth as a mode of intervention delivery or improving effectiveness.
The small but significant effect at follow-up suggests that eHealth interventions might have sustained positive effects on mental health. However, only half of the studies assessed follow-up outcomes, which may reflect a reporting bias. Supporting this, null, and in some cases, even negative, effects on mental health were observed in 27% (20/75) of the studies, and a greater proportion—60% (45/75)—did not report follow-up outcomes.
Less than one-fifth of all the trials used active controls. However, we found no significant differences in the efficacy of interventions tested against active versus passive controls overall or for any individual mental health outcome (). These findings do not confirm the digital placebo effect, which has been found in other eHealth reviews in clinical populations, where the ESs were lower in trials that used active controls than in those that used passive controls [ ]. Given that most (47/75, 63%) of these trials were not conducted in clinically unwell populations but were delivered universally to employees, this contrary finding may have implications for the types of control recommended for different settings of eHealth trials and for future framework analysis or guidelines.
The primary moderator of efficacy appeared to be the content of the intervention, with stress management and mindfulness-based interventions being seemingly more efficacious than CBT-based interventions. This suggests that CBT, which was adapted from its intended use in a more personalized and clinical setting, may have less useful content, especially for use in organizations with universal delivery, than other approaches adapted for more universal or indicated tools such as mindfulness and stress management.
In our previous review, stronger effects were seen in eHealth interventions that were supported , and other studies have found increased effects when eHealth interventions were supported by trained mental health professionals [ ]. The positive impacts of supported interventions may suggest that even in generally subclinical populations, a combination of eHealth and adjunctive support is the most effective in reducing mental health symptoms in employees.
Previous research has pointed to the importance of tailoring eHealth interventions to match individual user needs, as mental health symptoms differ from person to person . However, this review did not support the notion that the tailoring of interventions provides any benefit to improving mental health. The limited evidence available suggests no greater efficacy when using tailored interventions than when using interventions delivered universally or for indicated or even unwell samples. In fact, 43% (3/7) of the tailored interventions showed potential for harm in at least one mental health outcome.
This study had some limitations. Significant heterogeneity was detected; however, this is not uncommon in this field or in meta-analytic research . Differences across studies should be considered when interpreting these findings. It must also be acknowledged that some of the follow-up analyses were underpowered and that the findings should be interpreted tentatively. Furthermore, this review did not include any gray literature, and some authors did not provide data. Finally, the individual study pooled data calculated using the Comprehensive Meta-Analysis by Biostat Inc are not presented in this manuscript. Instead, we report the raw data of the pre-post means, SDs, and sample sizes in .
As we were unable to detect that eHealth interventions are improving over time, to design effective eHealth interventions, a better understanding of the factors that may influence efficacy is required. This study did not consider engagement and adherence and only considered a small number of potential moderating factors for each intervention. A subsequent framework analysis model is being undertaken to establish the potential beneficial or harmful features of the different types of interventions. This is an important future direction and requires further in-depth analysis as eHealth interventions have the potential to offer a range of novel self-management tools for employees with clinical and subclinical mental health conditions.
Another recommendation is to develop a standard framework for eHealth interventions to best understand the features that have therapeutic benefits and those that may potentially cause harm. This will guide developers to ensure that eHealth interventions are designed in the most effective manner. A framework may address the significant heterogeneity within studies; if the development standards are regulated against this framework, the differences in the features of the interventions and how they are delivered may be reduced. This research suggests that those evaluating interventions or designing protocols should carefully consider the level of support provided when interpreting reported ESs.
This review and meta-analysis confirms that eHealth interventions have a small but positive impact overall on reducing mental health symptoms in employees. There was significant heterogeneity between trials; however, overall, stress management and mindfulness interventions comprising in-person support appeared to be the most effective. Organizations should carefully consider the interventions delivered within the workplace; otherwise, they may not see long-term value in their return on investments. A substantial minority of intervention trials have demonstrated no efficacy, and a few may even be harmful. There is not enough evidence to make recommendations for the preferential use or development of therapeutically tailored interventions. Disappointingly, we found no evidence that despite the advancements in technology and the enormous resources in time, research, and finance devoted to this area for over a decade, better interventions are being produced. Hopefully, these small ESs do not represent the optimum outcome in organizational settings, and stakeholders and researchers should focus on improving effectiveness and efficacy or comparing and understanding the effects of current interventions. Despite the small effectiveness, this level of reduction could result in a large economic value in an organizational setting.
ES contributed to the conceptualization, data curation, formal analysis, investigation, methodology, validation, visualization, writing of the original draft, and review and editing of the manuscript. AL and IC contributed to data curation, methodology, and review and editing of the manuscript. IC contributed to data curation, methodology, and review and editing of the manuscript. HMG contributed to the data curation and review and editing of the manuscript. MA contributed to screening and study identification. JT contributed to the methodology, analysis plan, and review and editing of the manuscript. RAC and SMH contributed to the methodology and review and editing of the manuscript. NG contributed to data curation, investigation, methodology, supervision, and review and editing of the manuscript.
Conflicts of Interest
NG has a patent PCT/AU2019/051186 pending. NG and ES own the IP of a web-based disclosure decision aid tool for employees.
Example of search terms.DOCX File , 38 KB
References of the included studies.DOCX File , 71 KB
Estimated risk of bias.PNG File , 278 KB
Mental health measures.DOCX File , 28 KB
Detailed description of the identified studies and results.DOCX File , 101 KB
Overall studies small study effect.PNG File , 51 KB
- Harvey SB, Modini M, Joyce S, Milligan-Saville JS, Tan L, Mykletun A, et al. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup Environ Med 2017 Mar;74(4):301-310. [CrossRef] [Medline]
- Glozier N. Mental ill health and fitness for work. Occup Environ Med 2002 Oct;59(10):714-720 [FREE Full text] [CrossRef] [Medline]
- Harvey SB, Henderson M, Lelliott P, Hotopf M. Mental health and employment: much work still to be done. Br J Psychiatry 2009 Mar;194(3):201-203. [CrossRef] [Medline]
- Leka S, Jain A, Iavicoli S, Vartia M, Ertel M. The role of policy for the management of psychosocial risks at the workplace in the European Union. Saf Sci 2011 Apr;49(4):558-564. [CrossRef]
- Stevenson D, Farmer P. Thriving at Work: a review of mental health and employers. United Kingdom Government. 2017 Oct 26. URL: https://www.gov.uk/government/publications/thriving-at-work-a-review-of-mental-health-and-employers [accessed 2022-10-10]
- Knudsen AK, Harvey SB, Mykletun A, Øverland S. Common mental disorders and long-term sickness absence in a general working population. The Hordaland Health Study. Acta Psychiatr Scand 2013 Apr;127(4):287-297. [CrossRef] [Medline]
- Knudsen AK, Øverland S, Aakvaag HF, Harvey SB, Hotopf M, Mykletun A. Common mental disorders and disability pension award: seven year follow-up of the HUSK study. J Psychosom Res 2010 Jul;69(1):59-67. [CrossRef] [Medline]
- Harvey SB, Joyce S, Tan L, Johnson A, Nguyen H, Modini M, et al. Developing a mentally healthy workplace: a review of the literature – a report for the National Mental Health Commission and the Mentally Healthy Workplace Alliance. University of New South Wales. 2014 Nov. URL: https://www.headsup.org.au/docs/default-source/resources/developing-a-mentally-healthy-workplace_final-november-2014.pdf?sfvrsn=8 [accessed 2022-10-10]
- Henderson M, Harvey SB, Overland S, Mykletun A, Hotopf M. Work and common psychiatric disorders. J R Soc Med 2011 May;104(5):198-207 [FREE Full text] [CrossRef] [Medline]
- Ebert DD, Kählke F, Buntrock C, Berking M, Smit F, Heber E, et al. A health economic outcome evaluation of an Internet-based mobile-supported stress management intervention for employees. Scand J Work Environ Health 2018 Mar 01;44(2):171-182 [FREE Full text] [CrossRef] [Medline]
- Can Tech Help Manage Mental Health in the Workplace? CorporateWellnessMagazine. 2018. URL: https://www.corporatewellnessmagazine.com/article/tech-manage-mental-health-workplace [accessed 2020-08-04]
- Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010 Oct 13;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
- Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, REEACT Team. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015 Nov 11;351:h5627 [FREE Full text] [CrossRef] [Medline]
- So M, Yamaguchi S, Hashimoto S, Sado M, Furukawa TA, McCrone P. Is computerised CBT really helpful for adult depression?-A meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity. BMC Psychiatry 2013 Apr 15;13:113 [FREE Full text] [CrossRef] [Medline]
- Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
- Spijkerman MP, Pots WT, Bohlmeijer ET. Effectiveness of online mindfulness-based interventions in improving mental health: a review and meta-analysis of randomised controlled trials. Clin Psychol Rev 2016 Apr;45:102-114 [FREE Full text] [CrossRef] [Medline]
- McIntyre RS, Florea I, Tonnoir B, Loft H, Lam RW, Christensen MC. Efficacy of vortioxetine on cognitive functioning in working patients with major depressive disorder. J Clin Psychiatry 2017 Jan;78(1):115-121 [FREE Full text] [CrossRef] [Medline]
- Donkin L, Glozier N. Motivators and motivations to persist with online psychological interventions: a qualitative study of treatment completers. J Med Internet Res 2012 Jun 22;14(3):e91 [FREE Full text] [CrossRef] [Medline]
- Stratton E, Lampit A, Choi I, Calvo RA, Harvey SB, Glozier N. Effectiveness of eHealth interventions for reducing mental health conditions in employees: a systematic review and meta-analysis. PLoS One 2017 Dec 21;12(12):e0189904 [FREE Full text] [CrossRef] [Medline]
- Carolan S, Harris PR, Cavanagh K. Improving employee well-being and effectiveness: systematic review and meta-analysis of Web-based psychological interventions delivered in the workplace. J Med Internet Res 2017 Jul 26;19(7):e271 [FREE Full text] [CrossRef] [Medline]
- Phillips EA, Gordeev VS, Schreyögg J. Effectiveness of occupational e-mental health interventions: a systematic review and meta-analysis of randomized controlled trials. Scand J Work Environ Health 2019 Nov 01;45(6):560-576 [FREE Full text] [CrossRef] [Medline]
- Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 2016 Feb;3(2):100-102. [CrossRef] [Medline]
- Kauer SD, Reid SC, Crooke AH, Khor A, Hearps SJ, Jorm AF, et al. Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012 Jun 25;14(3):e67 [FREE Full text] [CrossRef] [Medline]
- Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [CrossRef] [Medline]
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
- Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. London, UK: The Cochrane Collaboration; Jan 01, 2013.
- Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019 Aug 28;366:l4898. [CrossRef] [Medline]
- Hedges L, Olkin I. Statistical Methods for Meta-Analysis. New York, NY, USA: Academic Press; 1985.
- Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons; 2009.
- Cumpston M, Li T, Page MJ, Chandler J, Welch VA, Higgins JP, et al. Updated guidance for trusted systematic reviews: a new edition of the Cochrane Handbook for Systematic Reviews of Interventions. Cochrane Database Syst Rev 2019 Oct 03;10:ED000142. [CrossRef] [Medline]
- Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002 Jun 15;21(11):1539-1558. [CrossRef] [Medline]
- Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 2010 Apr;1(2):97-111. [CrossRef] [Medline]
- Rothstein HR, Sutton AJ, Borenstein M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. Chichester, UK: John Wiley & Sons; 2005.
- Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000 Jun;56(2):455-463. [CrossRef] [Medline]
- Griffiths KM, Bennett K, Walker J, Goldsmid S, Bennett A. Effectiveness of MH-Guru, a brief online mental health program for the workplace: a randomised controlled trial. Internet Interv 2016 Nov;6:29-39 [FREE Full text] [CrossRef] [Medline]
- Guille C, Zhao Z, Krystal J, Nichols B, Brady K, Sen S. Web-based cognitive behavioral therapy intervention for the prevention of suicidal ideation in medical interns: a randomized clinical trial. JAMA Psychiatry 2015 Dec;72(12):1192-1198 [FREE Full text] [CrossRef] [Medline]
- Imamura K, Furukawa TA, Matsuyama Y, Shimazu A, Kuribayashi K, Kasai K, et al. Differences in the effect of Internet-based cognitive behavioral therapy for improving nonclinical depressive symptoms among workers by time preference: randomized controlled trial. J Med Internet Res 2018 Aug 10;20(8):e10231 [FREE Full text] [CrossRef] [Medline]
- Ketelaar SM, Nieuwenhuijsen K, Gärtner FR, Bolier L, Smeets O, Sluiter JK. Effect of an E-mental health approach to workers' health surveillance versus control group on work functioning of hospital employees: a cluster-RCT. PLoS One 2013 Sep 12;8(9):e72546 [FREE Full text] [CrossRef] [Medline]
- Volker D, Zijlstra-Vlasveld MC, Anema JR, Beekman AT, Brouwers EP, Emons WH, et al. Effectiveness of a blended Web-based intervention on return to work for sick-listed employees with common mental disorders: results of a cluster randomized controlled trial. J Med Internet Res 2015 May 13;17(5):e116 [FREE Full text] [CrossRef] [Medline]
- Cavanagh K, Churchard A, O'Hanlon P, Mundy T, Votolato P, Jones F, et al. A randomised controlled trial of a brief online mindfulness-based intervention in a non-clinical population: replication and extension. Mindfulness (N Y) 2018;9(4):1191-1205 [FREE Full text] [CrossRef] [Medline]
- Villani D, Grassi A, Cognetta C, Cipresso P, Toniolo D, Riva G. The effects of a mobile stress management protocol on nurses working with cancer patients: a preliminary controlled study. Stud Health Technol Inform 2012;173:524-528. [Medline]
- Geraedts AS, Kleiboer AM, Twisk J, Wiezer NM, van Mechelen W, Cuijpers P. Long-term results of a Web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res 2014 Jul 09;16(7):e168 [FREE Full text] [CrossRef] [Medline]
- Geraedts AS, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P. Short-term effects of a Web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res 2014 May 06;16(5):e121 [FREE Full text] [CrossRef] [Medline]
- Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Effects of an Internet-based cognitive behavioral therapy (iCBT) program in Manga format on improving subthreshold depressive symptoms among healthy workers: a randomized controlled trial. PLoS One 2014 May 20;9(5):e97167 [FREE Full text] [CrossRef] [Medline]
- Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Does Internet-based cognitive behavioral therapy (iCBT) prevent major depressive episode for workers? A 12-month follow-up of a randomized controlled trial. Psychol Med 2015 Jul;45(9):1907-1917. [CrossRef] [Medline]
- Eisen KP, Allen GJ, Bollash M, Pescatello LS. Stress management in the workplace: a comparison of a computer-based and an in-person stress-management intervention. Comput Human Behav 2008 Mar;24(2):486-496. [CrossRef]
- Wolever RQ, Bobinet KJ, McCabe K, Mackenzie ER, Fekete E, Kusnick CA, et al. Effective and viable mind-body stress reduction in the workplace: a randomized controlled trial. J Occup Health Psychol 2012 Apr;17(2):246-258. [CrossRef] [Medline]
- Boß L, Lehr D, Schaub MP, Paz Castro R, Riper H, Berking M, et al. Efficacy of a Web-based intervention with and without guidance for employees with risky drinking: results of a three-arm randomized controlled trial. Addiction 2018 Apr;113(4):635-646 [FREE Full text] [CrossRef] [Medline]
- Carolan S, Harris PR, Greenwood K, Cavanagh K. Increasing engagement with an occupational digital stress management program through the use of an online facilitated discussion group: results of a pilot randomised controlled trial. Internet Interv 2017 Dec;10:1-11 [FREE Full text] [CrossRef] [Medline]
- Gollwitzer PM, Mayer D, Frick C, Oettingen G. Promoting the self-regulation of stress in health care providers: an Internet-based intervention. Front Psychol 2018 Jun 15;9:838 [FREE Full text] [CrossRef] [Medline]
- Mak WW, Chan AT, Cheung EY, Lin CL, Ngai KC. Enhancing Web-based mindfulness training for mental health promotion with the health action process approach: randomized controlled trial. J Med Internet Res 2015 Jan 19;17(1):e8 [FREE Full text] [CrossRef] [Medline]
- Shirotsuki K, Nonaka Y, Abe K, Adachi SI, Adachi S, Kuboki T, et al. The effect for Japanese workers of a self-help computerized cognitive behaviour therapy program with a supplement soft drink. Biopsychosoc Med 2017 Sep 19;11:23 [FREE Full text] [CrossRef] [Medline]
- Allexandre D, Bernstein AM, Walker E, Hunter J, Roizen MF, Morledge TJ. A Web-based mindfulness stress management program in a corporate call center: a randomized clinical trial to evaluate the added benefit of onsite group support. J Occup Environ Med 2016 Mar;58(3):254-264 [FREE Full text] [CrossRef] [Medline]
- Grime PR. Computerized cognitive behavioural therapy at work: a randomized controlled trial in employees with recent stress-related absenteeism. Occup Med (Lond) 2004 Aug;54(5):353-359. [CrossRef] [Medline]
- Hasson D, Anderberg UM, Theorell T, Arnetz BB. Psychophysiological effects of a Web-based stress management system: a prospective, randomized controlled intervention study of IT and media workers [ISRCTN54254861]. BMC Public Health 2005 Jul 25;5:78 [FREE Full text] [CrossRef] [Medline]
- Shimazu A, Kawakami N, Irimajiri H, Sakamoto M, Amano S. Effects of Web-based psychoeducation on self-efficacy, problem solving behavior, stress responses and job satisfaction among workers: a controlled clinical trial. J Occup Health 2005 Sep;47(5):405-413 [FREE Full text] [CrossRef] [Medline]
- Cook RF, Billings DW, Hersch RK, Back AS, Hendrickson A. A field test of a Web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity: randomized controlled trial. J Med Internet Res 2007 Jun 19;9(2):e17 [FREE Full text] [CrossRef] [Medline]
- Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B. E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther 2007;36(3):179-192. [CrossRef] [Medline]
- Billings DW, Cook RF, Hendrickson A, Dove DC. A Web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med 2008 Aug;50(8):960-968 [FREE Full text] [CrossRef] [Medline]
- Suzuki E, Tsuchiya M, Hirokawa K, Taniguchi T, Mitsuhashi T, Kawakami N. Evaluation of an Internet-based self-help program for better quality of sleep among Japanese workers: a randomized controlled trial. J Occup Health 2008 Oct 20;50(5):387-399 [FREE Full text] [CrossRef] [Medline]
- Yamagishi M, Kobayashi T, Nakamura Y. Effects of web-based career identity training for stress management among Japanese nurses: a randomized control trial. J Occup Health 2008;50(2):191-193 [FREE Full text] [CrossRef] [Medline]
- Abbott JA, Klein B, Hamilton C, Rosenthal A. The impact of online resilience training for sales managers on wellbeing and performance. E J Appl Psychol 2009 Jun 15;5(1):89-95. [CrossRef]
- Bennett JB, Broome KM, Schwab-Pilley A, Gilmore P. A Web-based approach to address cardiovascular risks in managers: results of a randomized trial. J Occup Environ Med 2011 Aug;53(8):911-918 [FREE Full text] [CrossRef] [Medline]
- Glück TM, Maercker A. A randomized controlled pilot study of a brief Web-based mindfulness training. BMC Psychiatry 2011 Nov 08;11:175 [FREE Full text] [CrossRef] [Medline]
- Borness C, Proudfoot J, Crawford J, Valenzuela M. Putting brain training to the test in the workplace: a randomized, blinded, multisite, active-controlled trial. PLoS One 2013;8(3):e59982 [FREE Full text] [CrossRef] [Medline]
- Feicht T, Wittmann M, Jose G, Mock A, von Hirschhausen E, Esch T. Evaluation of a seven-week Web-based happiness training to improve psychological well-being, reduce stress, and enhance mindfulness and flourishing: a randomized controlled occupational health study. Evid Based Complement Alternat Med 2013;2013:676953 [FREE Full text] [CrossRef] [Medline]
- Lappalainen P, Kaipainen K, Lappalainen R, Hoffrén H, Myllymäki T, Kinnunen ML, et al. Feasibility of a personal health technology-based psychological intervention for men with stress and mood problems: randomized controlled pilot trial. JMIR Res Protoc 2013 Jan 09;2(1):e1 [FREE Full text] [CrossRef] [Medline]
- Villani D, Grassi A, Cognetta C, Toniolo D, Cipresso P, Riva G. Self-help stress management training through mobile phones: an experience with oncology nurses. Psychol Serv 2013 Aug;10(3):315-322. [CrossRef] [Medline]
- Bolier L, Ketelaar SM, Nieuwenhuijsen K, Smeets O, Gärtner FR, Sluiter JK. Workplace mental health promotion online to enhance well-being of nurses and allied health professionals: a cluster-randomized controlled trial. Internet Interv 2014 Oct;1(4):196-204. [CrossRef]
- Deitz D, Cook RF, Hersch RK, Leaf S. Heart healthy online: an innovative approach to risk reduction in the workplace. J Occup Environ Med 2014 May;56(5):547-553 [FREE Full text] [CrossRef] [Medline]
- Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: a randomized controlled trial. Internet Interv 2014 Jul;1(3):95-101. [CrossRef]
- Mori M, Tajima M, Kimura R, Sasaki N, Somemura H, Ito Y, et al. A Web-based training program using cognitive behavioral therapy to alleviate psychological distress among employees: randomized controlled pilot trial. JMIR Res Protoc 2014 Dec 02;3(4):e70 [FREE Full text] [CrossRef] [Medline]
- Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med 2014 Mar;44(4):741-752 [FREE Full text] [CrossRef] [Medline]
- Umanodan R, Shimazu A, Minami M, Kawakami N. Effects of computer-based stress management training on psychological well-being and work performance in japanese employees: a cluster randomized controlled trial. Ind Health 2014;52(6):480-491 [FREE Full text] [CrossRef] [Medline]
- Carissoli C, Villani D, Riva G. Does a meditation protocol supported by a mobile application help people reduce stress? Suggestions from a controlled pragmatic trial. Cyberpsychol Behav Soc Netw 2015 Jan;18(1):46-53. [CrossRef] [Medline]
- Cook RF, Hersch RK, Schlossberg D, Leaf SL. A Web-based health promotion program for older workers: randomized controlled trial. J Med Internet Res 2015 Mar 25;17(3):e82 [FREE Full text] [CrossRef] [Medline]
- Ebert DD, Berking M, Thiart H, Riper H, Laferton JA, Cuijpers P, et al. Restoring depleted resources: efficacy and mechanisms of change of an Internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol 2015 Dec;34S:1240-1251. [CrossRef] [Medline]
- Prasek A. Randomized Controlled Trial to Evaluate a Self-Guided, Web-Based Mindfulness Program for Stress Reduction and Wellbeing Promotion. University of Minnesota. 2015 Aug. URL: https://core.ac.uk/download/pdf/76357635.pdf [accessed 2022-08-29]
- Stansfeld SA, Kerry S, Chandola T, Russell J, Berney L, Hounsome N, et al. Pilot study of a cluster randomised trial of a guided e-learning health promotion intervention for managers based on management standards for the improvement of employee well-being and reduction of sickness absence: GEM Study. BMJ Open 2015 Oct 26;5(10):e007981 [FREE Full text] [CrossRef] [Medline]
- Yuan Q. Evaluating the Effectiveness of A Psychological Capital Development Program on Mental Health, Engagement and Work Performance. The Chinese University of Hong Kong. 2015. URL: https://www.proquest.com/docview/1846509447?pq-origsite=gscholar&fromopenview=true [accessed 2021-10-10]
- Birney AJ, Gunn R, Russell JK, Ary DV. MoodHacker mobile Web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth 2016 Jan 26;4(1):e8 [FREE Full text] [CrossRef] [Medline]
- Bostock S, Crosswell AD, Prather AA, Steptoe A. Mindfulness on-the-go: effects of a mindfulness meditation app on work stress and well-being. J Occup Health Psychol 2019 Feb;24(1):127-138 [FREE Full text] [CrossRef] [Medline]
- Dyrbye LN, West CP, Richards ML, Ross HJ, Satele D, Shanafelt TD. A randomized, controlled study of an online intervention to promote job satisfaction and well-being among physicians. Burnout Res 2016 Sep;3(3):69-75. [CrossRef]
- Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, et al. Self-guided Internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med 2016 May;73(5):315-323. [CrossRef] [Medline]
- Ebert DD, Lehr D, Boß L, Riper H, Cuijpers P, Andersson G, et al. Efficacy of an Internet-based problem-solving training for teachers: results of a randomized controlled trial. Scand J Work Environ Health 2014 Nov;40(6):582-596 [FREE Full text] [CrossRef] [Medline]
- Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res 2016 Jan 27;18(1):e21 [FREE Full text] [CrossRef] [Medline]
- Hersch RK, Cook RF, Deitz DK, Kaplan S, Hughes D, Friesen MA, et al. Reducing nurses' stress: a randomized controlled trial of a web-based stress management program for nurses. Appl Nurs Res 2016 Nov;32:18-25 [FREE Full text] [CrossRef] [Medline]
- Jonas B, Leuschner F, Tossmann P. Efficacy of an Internet-based intervention for burnout: a randomized controlled trial in the German working population. Anxiety Stress Coping 2017 Mar;30(2):133-144. [CrossRef] [Medline]
- Beiwinkel T, Eißing T, Telle N, Siegmund-Schultze E, Rössler W. Effectiveness of a Web-based intervention in reducing depression and sickness absence: randomized controlled trial. J Med Internet Res 2017 Jun 15;19(6):e213 [FREE Full text] [CrossRef] [Medline]
- Zhang H, Jiang Y, Nguyen HD, Poo DC, Wang W. The effect of a smartphone-based coronary heart disease prevention (SBCHDP) programme on awareness and knowledge of CHD, stress, and cardiac-related lifestyle behaviours among the working population in Singapore: a pilot randomised controlled trial. Health Qual Life Outcomes 2017 Mar 14;15(1):49 [FREE Full text] [CrossRef] [Medline]
- Zwerenz R, Becker J, Gerzymisch K, Siepmann M, Holme M, Kiwus U, et al. Evaluation of a transdiagnostic psychodynamic online intervention to support return to work: a randomized controlled trial. PLoS One 2017 May 8;12(5):e0176513 [FREE Full text] [CrossRef] [Medline]
- Bostock S, Luik AI, Espie CA. Sleep and productivity benefits of digital cognitive behavioral therapy for insomnia: a randomized controlled trial conducted in the workplace environment. J Occup Environ Med 2016 Jul;58(7):683-689. [CrossRef] [Medline]
- Eriksson T, Germundsjö L, Åström E, Rönnlund M. Mindful self-compassion training reduces stress and burnout symptoms among practicing psychologists: a randomized controlled trial of a brief Web-based intervention. Front Psychol 2018 Nov 27;9:2340 [FREE Full text] [CrossRef] [Medline]
- Hamamura T, Suganuma S, Ueda M, Mearns J, Shimoyama H. Standalone effects of a cognitive behavioral intervention using a mobile phone app on psychological distress and alcohol consumption among Japanese workers: pilot nonrandomized controlled trial. JMIR Ment Health 2018 Mar 22;5(1):e24 [FREE Full text] [CrossRef] [Medline]
- Lilly M, Calhoun R, Painter I, Beaton R, Stangenes S, Revere D, et al. Destress 9-1-1-an online mindfulness-based intervention in reducing stress among emergency medical dispatchers: a randomised controlled trial. Occup Environ Med 2019 Oct;76(10):705-711. [CrossRef] [Medline]
- Mistretta EG, Davis MC, Temkit M, Lorenz C, Darby B, Stonnington CM. Resilience training for work-related stress among health care workers: results of a randomized clinical trial comparing in-person and smartphone-delivered interventions. J Occup Environ Med 2018 Jun;60(6):559-568. [CrossRef] [Medline]
- Oishi S, Takizawa T, Kamata N, Miyaji S, Tanaka K, Miyaoka H. Web-based training program using cognitive behavioral therapy to enhance cognitive flexibility and alleviate psychological distress among schoolteachers: pilot randomized controlled trial. JMIR Res Protoc 2018 Jan 26;7(1):e32 [FREE Full text] [CrossRef] [Medline]
- Persson Asplund R, Dagöö J, Fjellström I, Niemi L, Hansson K, Zeraati F, et al. Internet-based stress management for distressed managers: results from a randomised controlled trial. Occup Environ Med 2018 Feb;75(2):105-113 [FREE Full text] [CrossRef] [Medline]
- Querstret D, Cropley M, Fife-Schaw C. The effects of an online mindfulness intervention on perceived stress, depression and anxiety in a non-clinical sample: a randomised waitlist control trial. Mindfulness (N Y) 2018;9(6):1825-1836 [FREE Full text] [CrossRef] [Medline]
- Song M, Kanaoka H. Effectiveness of mobile application for menstrual management of working women in Japan: randomized controlled trial and medical economic evaluation. J Med Econ 2018 Nov;21(11):1131-1138 [FREE Full text] [CrossRef] [Medline]
- Coelhoso CC, Tobo PR, Lacerda SS, Lima AH, Barrichello CR, Amaro E, et al. A new mental health mobile app for well-being and stress reduction in working women: randomized controlled trial. J Med Internet Res 2019 Nov 07;21(11):e14269 [FREE Full text] [CrossRef] [Medline]
- Stratton E, Choi I, Calvo R, Hickie I, Henderson C, Harvey SB, et al. Web-based decision aid tool for disclosure of a mental health condition in the workplace: a randomised controlled trial. Occup Environ Med 2019 Sep;76(9):595-602. [CrossRef] [Medline]
- Weber S, Lorenz C, Hemmings N. Improving stress and positive mental health at work via an app-based intervention: a large-scale multi-center randomized control trial. Front Psychol 2019 Dec 6;10:2745 [FREE Full text] [CrossRef] [Medline]
- Deady M, Choi I, Calvo RA, Glozier N, Christensen H, Harvey SB. eHealth interventions for the prevention of depression and anxiety in the general population: a systematic review and meta-analysis. BMC Psychiatry 2017 Aug 29;17(1):310 [FREE Full text] [CrossRef] [Medline]
- Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
- Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012 Mar;18(2):137-144. [CrossRef] [Medline]
|CBT: cognitive behavioral therapy|
|ES: effect size|
|PICOS: Patient/Population, Intervention, Comparison, and Outcomes|
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|PROSPERO: International Prospective Register of Systematic Reviews|
|RCT: randomized controlled trial|
|SMD: standardized mean difference|
Edited by R Kukafka; submitted 07.03.22; peer-reviewed by E Phillips, L Shen, V Nagy; comments to author 18.05.22; revised version received 26.05.22; accepted 29.07.22; published 27.09.22Copyright
©Elizabeth Stratton, Amit Lampit, Isabella Choi, Hanna Malmberg Gavelin, Melissa Aji, Jennifer Taylor, Rafael A Calvo, Samuel B Harvey, Nick Glozier. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 27.09.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.