JMIR Publications

Journal of Medical Internet Research

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 26.07.17 in Vol 19, No 7 (2017): July

This paper is in the following e-collection/theme issue:

    Original Paper

    Improving Employee Well-Being and Effectiveness: Systematic Review and Meta-Analysis of Web-Based Psychological Interventions Delivered in the Workplace

    School of Psychology, University of Sussex, Brighton, United Kingdom

    Corresponding Author:

    Stephany Carolan, MSc

    School of Psychology

    University of Sussex

    Falmer

    Brighton, BN1 9QH

    United Kingdom

    Phone: 44 (0)1273 876638

    Fax:44 (0)1273 678058

    Email:


    ABSTRACT

    Background: Stress, depression, and anxiety among working populations can result in reduced work performance and increased absenteeism. Although there is evidence that these common mental health problems are preventable and treatable in the workplace, uptake of psychological treatments among the working population is low. One way to address this may be the delivery of occupational digital mental health interventions. While there is convincing evidence for delivering digital psychological interventions within a health and community context, there is no systematic review or meta-analysis of these interventions in an occupational setting.

    Objective: The aim of this study was to identify the effectiveness of occupational digital mental health interventions in enhancing employee psychological well-being and increasing work effectiveness and to identify intervention features associated with the highest rates of engagement and adherence.

    Methods: A systematic review of the literature was conducted using Cochrane guidelines. Papers published from January 2000 to May 2016 were searched in the PsychINFO, MEDLINE, PubMed, Science Direct, and the Cochrane databases, as well as the databases of the researchers and relevant websites. Unpublished data was sought using the Conference Proceedings Citation Index and the Clinical Trials and International Standard Randomized Controlled Trial Number (ISRCTN) research registers. A meta-analysis was conducted by applying a random-effects model to assess the pooled effect size for psychological well-being and the work effectiveness outcomes. A positive deviance approach was used to identify those intervention features associated with the highest rates of engagement and adherence.

    Results: In total, 21 randomized controlled trials (RCTs) met the search criteria. Occupational digital mental health interventions had a statistically significant effect post intervention on both psychological well-being (g=0.37, 95% CI 0.23-0.50) and work effectiveness (g=0.25, 95% CI 0.09-0.41) compared with the control condition. No statistically significant differences were found on either outcome between studies using cognitive behavioral therapy (CBT) approaches (as defined by the authors) compared with other psychological approaches, offering guidance compared with self-guidance, or recruiting from a targeted workplace population compared with a universal workplace population. In-depth analysis of the interventions identified by the positive deviance approach suggests that interventions that offer guidance are delivered over a shorter time frame (6 to 7 weeks), utilize secondary modalities for delivering the interventions and engaging users (ie, emails and text messages [short message service, SMS]), and use elements of persuasive technology (ie, self-monitoring and tailoring), which may achieve greater engagement and adherence.

    Conclusions: This review provides evidence that occupational digital mental health interventions can improve workers’ psychological well-being and increase work effectiveness. It identifies intervention characteristics that may increase engagement. Recommendations are made for future research, practice, and intervention development.

    J Med Internet Res 2017;19(7):e271

    doi:10.2196/jmir.7583

    KEYWORDS



    Introduction

    Background

    Nearly one in three workers in Europe [1] and the United States [2] report that they are affected by stress at work. Work-related stress, depression, and anxiety can result in reduced work performance and absenteeism [3-7], costing an estimated 3% to 4% of gross national product [1]. There is evidence that these conditions are both preventable and treatable in the workplace [8-9] and that workers who receive treatment are more likely to be highly productive [10,11].

    The workplace has been identified as a potentially ideal site for delivering mental health prevention programs [12] and increasing access to appropriate treatment [7], resulting in a benefit to both employees and employers [11]. However, uptake of psychological treatments among the working population is low [10], with one study reporting that only 15% of workers with a mental health problem had sought help in the preceding month [13], resulting in many depressed workers going untreated or being inadequately treated [11]. Help seeking among the working population has been reported at between 43% [10] and 15% [13]. People are increasingly turning to the Internet for health care information [14], prevention, and treatment [15]. Although there is convincing empirical evidence for the effectiveness of evidence-based digital psychological interventions delivered within a health and community context, the evidence for digital interventions delivered in a workplace setting is less clear [16].

    Several meta-analyses and systematic reviews have found evidence for the effectiveness of digital psychological interventions delivered in nonworkplace settings for common mental health problems including depression, anxiety [17-23], and stress in adults [24], but these reviews do not focus on the delivery of these interventions to working adults or in the workplace. We suggest that the delivery of occupational health interventions is different to the delivery of interventions in health or community settings and that the context of the workplace is likely to impact on the way that these interventions are delivered and received, and is therefore, likely to impact on their effectiveness. To our knowledge no previous systematic review has specifically reported on digital interventions for stress and mental health in the workplace.

    This systematic review and meta-analysis seeks to address this gap in the literature by identifying studies that deliver digital occupational mental health interventions and evaluating their effectiveness at increasing employee psychological well-being (by targeting a reduction in stress, depression, and psychological distress) and work effectiveness.

    Engagement and adherence are two of the major challenges to delivering and evaluating Web-based interventions [25-27]. Boosting engagement and adherence with Web-based interventions increases the extent to which users are exposed to the content and may be an important determinant of effectiveness [28] and a consistent predictor of positive outcomes [29-31].

    This review uses a positive deviance approach (eg, [32,33]) to identify the intervention features that are associated with the highest levels of intervention engagement and adherence in the workplace context.

    Aims of This Review

    The aims of this systematic review and meta-analysis are to evaluate the overall effectiveness of occupational digital mental health interventions for employee psychological well-being and work effectiveness and to identify, through the partial implementation of positive deviance methodology, which intervention features influence engagement and adherence. To this end, the review will address the following three questions:

    1. Are occupational digital mental health interventions associated with lower levels of stress and mental health symptoms post intervention than control groups?
    2. Are occupational digital mental health interventions associated with increased work effectiveness post intervention?
    3. Which intervention features are associated with the highest levels of engagement and adherence?

    Methods

    Search Strategy

    This review was conducted following the Cochrane guidance for systematic reviews [34]. We searched PsychINFO, MEDLINE, PubMed, Science Direct, and the Cochrane database of systematic reviews for relevant studies published from January 2000 to May 2016. The key terms used for these searches are displayed in Table 1. To increase coverage, we searched the databases of the researchers, relevant websites (eg, the Health and Safety Executive, the Faculty of Occupational Medicine, and the National Institute for Heath and Care Excellence), reference lists of included studies, and relevant journals. Unpublished data was sought using the Conference Proceedings Citation Index and the Clinical Trials and ISRCTN research registers. Three potentially relevant trials were identified through the research registers, and the researchers were contacted. However, no additional data from these unpublished studies became available.

    Table 1. Search terms.
    View this table

    Inclusion and Exclusion Criteria

    To meet the aims of this review, a study had to meet the following criteria: (1) use a randomized controlled design; (2) utilize a nontreatment, treatment as usual, or active control; (3) aimed at employed participants aged 18 years or over; (4) comprise a psychological intervention aimed at increasing psychological well-being (eg, by reducing symptoms of stress or depression) or work effectiveness (eg, by increasing engagement or productivity); (5) be delivered via the Internet, mobile technology, or a computer program; (6) written in English; and (7) offer sufficient post intervention data (sample sizes, means, and standard deviations [SDs] for both the control and the treatment condition) in the paper or by contacting the authors to calculate the effect size for either a well-being or a work effectiveness outcome.

    Studies were excluded if they exclusively targeted people on extended sick leave or were targeting populations with complex mental health problems including post-traumatic stress disorder (PTSD), schizophrenia, or comorbid substance misuse. Studies were also excluded if technology was used purely as a medium for communication (eg, Skype, videoconference, e-counseling): the active element of the intervention had to be delivered on the Web or via mobile technology. Studies were also excluded if homework was completed on the Web but the intervention was delivered in person.

    Data Extraction

    The data was coded at four levels: study, intervention, participants, and outcomes. Further information about coding categories is available from the study registration (the protocol for this systematic review and meta-analysis was registered with the International Prospective Register of Systematic Reviews (PROSPERO; registration number CRD42016033935).

    Data Analysis

    The number of participants and the between group, post intervention means, and SD for the control and the experimental group on selected psychological measures (measures prioritized in the order: stress, depression, and psychological distress) and selected work effectiveness measures (prioritized in the order: work engagement, productivity or job specific effectiveness, work related self-efficacy, and work related rumination) were entered into Review Manager (RevMan) version 5.3 and SPSS version 22 (IBM Corp). Where more than one measure was available, the measures were prioritized in the order given above. Forest plots of the between group, post intervention effect size (Hedges g) for both outcome variables (psychological well-being and work effectiveness) were produced using RevMan. The magnitude of effect size was interpreted using the classification given by Cohen (small=0.2, medium=0.5, and large=0.8) [35].

    To test for the presence of heterogeneity of effect size, we used the chi-square (χ2) and the heterogeneity (I2) statistics. A large χ2 relative to its degree of freedom and a low P value provides evidence of heterogeneity [34]. An I2 value of 25% suggests that heterogeneity is low, 50% suggests medium, and 75% suggests high [36]. Since we expected considerable heterogeneity, a random effects model was performed [37]. Heterogeneity was explored using subgroup analyses. Possible moderating factors included (1) therapeutic approach (cognitive behavioral therapy [CBT] vs other), (2) guidance (guided vs nonguided), and (3) population (targeted vs universal). Interventions were coded as using CBT if the authors of the studies described the therapeutic approach as cognitive or cognitive behavior and as guided if guidance from a person was described. We coded the population as targeted if the inclusion criteria included elevated levels of stress, depression, or insomnia. Publication bias was tested using funnel plots for both outcome measures.

    Risk of Bias Assessment

    An assessment of the methodological quality of the studies included in this review was conducted using the Cochrane Collaboration’s risk of bias tool [34]. The tool assesses possible sources of bias using seven main categories: (1) random sequence generation, (2) allocation concealment, (3) blinding of participants and personnel, (4) blinding of outcome assessment, (5) incomplete outcome data, (6) selective reporting, and (7) other bias. Twenty-five percent of studies were assessed by the first and second author independently, with a high rate of agreement; differences were discussed and resolved. The first author completed all subsequent bias assessments. Publication bias was assessed by appraising funnel plots for asymmetry.

    Positive Deviance

    A partial implementation of the positive deviance approach was used to identify intervention features associated with the highest levels of engagement and adherence. Positive deviance is as an assets-based approach used to identify sustainable solutions to difficult problems by identifying “uncommon, beneficial practices” [33]. Bradley et al [32] describe four steps to using the positive deviance approach: (1) identify “positive deviants,” that is, organizations that consistently demonstrate exceptionally high performance in the area of interest, (2) study the organizations in depth to generate hypotheses about practices that enable organizations to achieve high performance, (3) test hypotheses with other organizations, and (4) work with other organizations to disseminate the evidence about high performance. In this study, the first two steps were adapted and applied to study interventions showing the highest levels of engagement (cf. [38]). To assess engagement, we ranked the 21 studies in this review in percentile order in terms of intervention completion and intervention group study attrition. Completion of the intervention and intervention group study attrition were seen as the most relevant and widely report measures of intervention engagement and adherence. Studies at the 70th percentile and above were selected and their interventions were reviewed in depth to generate hypotheses about intervention features that may enable high levels of engagement (hypotheses generation). This is a modification from our protocol.


    Results

    Search Results

    The initial search resulted in 1129 citations after duplicates had been removed. These citations were screened using the exclusion and inclusion criteria and 1076 excluded. Full papers were retrieved and examined for eligibility for the remaining 53 studies. We included 21 studies in the review: 21 in the qualitative synthesis, 21 in the psychological well-being meta-analysis, and 13 of the 21 in the work effectiveness meta-analysis. See Figure 1 for the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart of study selection. One study [39] did not exclude unemployed participants, but the aim of the study was to assess effectiveness of cognitive behavioral treatment for work related stress; 80% of the participants were in full time work and a number were unemployed because of work-related stress or were experiencing stress in unpaid jobs. For these reasons we included the study in the review. A sensitivity analysis indicated no difference in our overall results if this study was excluded. A second study [40] examined the effects on job stress of Web-based career identity training on Japanese hospital nurses. This study was excluded from the review as it was felt that the intervention was closer to a career counseling intervention than a psychological intervention.

    Designs of the Included Studies

    The 21 RCTs included in this review compared a Web-based psychological intervention delivered in the workplace with a wait list control (WLC) (71%, 15/21), an active control (19%, 4/21), or care as usual (9%, 2/21). Additionally, 17 (81%, 17/21) of the studies completed an intention-to-treat analysis, and 4 (19%, 4/21) completed a per-protocol analysis. Multimedia Appendix 1 describes the selected characteristics for the 21 identified studies.

    Risk of Bias

    Figure 2 shows an estimation of the risk of bias across all studies. Of the 21 studies included in this review, only 8 (38%) were able to fulfill 5 or more low risk of bias ratings across the seven categories used. Only 2 of the studies (9%, 2/21) were able to blind both participants and personnel to the condition allocation (performance bias), and only 6 (29%, 6/21) demonstrated low reporting bias by preregistering or making their study protocol available and by reporting all the primary outcomes. Less than half of all ratings (45.6%, 67/147) were unclear or high risk.

    Publication Bias

    Funnel plots for the effect sizes for the psychological wellbeing outcome and the work effectiveness outcome are shown in Figures 3 and 4, respectively. There is no indication of problematic clustering in these plots, which are fairly evenly distributed around the mean effect size, suggesting little evidence of publication bias.

    Figure 1. Flow diagram of study selection.
    View this figure
    Figure 2. Estimated risk of bias across all studies.
    View this figure
    Figure 3. Funnel plot of post intervention effect sizes by standard error for the psychological wellbeing outcome.
    View this figure
    Figure 4. Funnel plot of post intervention effect sizes by standard error for the work effectiveness outcome.
    View this figure

    Sample and Study Characteristics

    The 21 studies included in this review originated from 7 countries: 6 from the United States [41-46], 6 from Germany [47-52], 3 from the Netherlands [39,53,54], two each from the United Kingdom [55,56] and Japan [57,58], and one each from Australia [59] and Sweden [60]. Four of the studies recruited from the general working population [39,41,47,48], whereas the other studies recruited from organizations working in education [49-51], health, or local authorities [53,55,56]; a call center [42]; manufacturing [57]; technology [43,58]; sales [59]; chemicals [44]; human resource (HR) [45]; insurance [52]; and transport and communication [56]. One study recruited from organizations working in banking, research, education, and security [54]; 1 study recruited middle managers from medium and large companies [60], and another recruited employed care-givers of people with dementia [46].

    The sample size in the studies ranged from 48 to 762. Overall, the studies recruited and randomized 5260 participants: 2711 to a psychological intervention delivered predominantly on the Web and 2549 to a control condition. The data for 2438 participants was analyzed in the experimental group and 2360 in the control group. The discrepancy in numbers between randomized and analyzed is accounted for by study attrition [55,56,58].

    Women made up 58% (3051/5260) of all randomized participants. All the studies were aimed at a working age population. The range of mean ages reported across the studies was 36.4 to 48.4 years in the intervention groups and 34.3 to 47.8 years in the control groups. Nine of the studies (43%) recruited from a targeted population, including individuals with elevated levels of depression [41,54,56], stress [47-49], and insomnia [50,51]; one study recruited participants who had taken 10 or more consecutive days off work for stress, anxiety, or depression [55]. The remaining 12 studies (57%, 12/21) targeted a universal population with no set psychological inclusion criteria.

    Intervention Characteristics

    Multimedia Appendix 2 describes the selected characteristics of the interventions used in the studies included in this review. Over half of the interventions were based on cognitive or cognitive behaviour therapy (12/21, 57%) [39,41,43,50,51, 53-59], with 3 based on stress and coping (14%) [46-48], 2 on mindfulness (10%) [42,44], and one each (5%) on social cognitive theory [45], problem solving training [49], positive psychology [52], and acceptance and commitment therapy [60]. The mean duration of the interventions was 7.6 weeks (SD=2.5; range 4.3 to 13.0). Seventeen (81%) of the interventions included in the studies used a website as their primary means of delivering the intervention [42-54,56-59], 2 (10%) delivered the intervention via a computer application [41,60], 1 (5%) via email [39], and one (5%) through a standalone computer [55]. Secondary modalities used by the studies to deliver the intervention and to engage users were email (12/21, 57%) [41,42,44,46,48,51-54,57-59], texting (4/21, 19%) [44,47,48,60], conference calls (2/21, 10%) [44,59], telephone calls (1/21, 5%) [59], face-to-face delivery (1/21, 5%) [44], a workbook (1/21, 5%) [44], and a compact disc (CD; 1/21, 5%) [42]. Just over half of the interventions (11/21, 52%) were self-guided [41-43,45-47,50,52,53,55,56], and 10 (48%, 10/21) offered users of the intervention some form of guidance: seven of those 10 studies (70%) described the guidance as coming from a therapist or coach [39,44,48,49,51,54,60], 2 (20%) were described as a coordinator or member of staff [57,59], and one (10%) as a clinical psychologist [58].

    Study Attrition and Intervention Completion

    Study attrition for the control and the intervention groups separately was available for 20 of the studies (one study reported combined study attrition [46]). The mean attrition for the intervention groups was 23% (SD=16.1, range 3% to 54%) and for the control groups 13% (SD=11.6, range 0% to 41%).

    Intervention completion (adherence) data was available for 19 of the studies (data not available for 2 of the studies [41,52]). Most studies reported the percentage of participants that completed all or part of the intervention. The mean adherence (taken as the highest level of completion reported by the authors) was 45% (SD=29.3, range 3% to 95%).

    Persuasive Technology

    Studies were coded to see what if any elements of persuasive technology the interventions used to help support users to benefit from the intervention. They were coded using the classifications given by Fogg [61]. These are (1) reduction (reducing complex behavior to simple tasks), (2) tunneling (leading users through a predetermined sequence of actions or events), (3) tailoring (providing information relevant to specific individuals), (4) suggestion (making a suggestion at the most appropriate time), (5) self-monitoring (enabling people to monitor themselves), (6) surveillance (the use of computer technology to allow one party to monitor the behavior of another), and (7) conditioning (using technology to reinforce target behaviors). Seventeen of the 21 studies (81%) reported using a form of persuasive technology [39,41,43,44,46-51,53-55,57-60]. Tailoring was used by 57% (12/21) of interventions [39,43,44,46-51,53,54,58], self-monitoring by 43% (9/21) [41,44,47,50,51,55,57,59,60], and tunneling by 14% (3/21) [41,54,55]. We were unable to identify any forms of persuasive technology in the descriptions of 19% (4/21) of studies [42,45,52,56].

    Meta-Analyses Findings

    Post intervention means, SDs, and group numbers were extracted from the 21 studies included in this review. Two separate meta-analyses were completed for (1) psychological well-being, and (2) work effectiveness. Of the 21 studies included in the psychological well-being meta-analysis, 13 were also included in the work effectiveness meta-analysis. Both analyses were conducted using a random-effects model.

    Figure 5 is a forest plot for the 21 studies that included a measure of psychological well-being. The Web-based psychological intervention delivered in the workplace resulted in significantly reduced levels of stress, depression, and psychological distress scores post intervention for the intervention condition compared with the control condition (z20=5.24, P<.001) with a small effect size (g=0.37, 95% CI 0.23-0.50). The resulting effect sizes were significantly and highly heterogeneous (χ220=103.1 P<.001; I2=81%).

    Figure 6 shows a forest plot for the 13 studies that included a work effectiveness measure. Participants in the intervention group showed significantly greater workplace effectiveness scores compared with those in the control conditions (z12=3.00, P=.003) with a small effect size (g=0.25, 95% CI 0.09-0.41). The resulting effect sizes were significantly and highly heterogeneous (χ212=48.2, P<.001, I2= 75%).

    The results of both meta-analyses suggested that further subgroup analyses were warranted.

    Figure 5. Forest plot of post intervention effect size for the psychological wellbeing outcome.
    View this figure
    Figure 6. Forest plot of post intervention effect size for the work effectiveness outcome.
    View this figure

    Sensitivity Analysis

    We conducted outlier analysis by examining the forest plots of standard mean difference effect sizes and CIs for both the psychological well-being measure and the work effectiveness measure. One study was identified as a possible outlier on the psychological well-being outcome [59] because of its negative effect size (contrary to the other studies) and because its CIs did not fall into the range of the other studies. A sensitivity analysis excluding the study from analysis shows that the result of the main effect remains robust. A sensitivity analysis was also conducted comparing studies with lower and higher risk of bias. Studies with a low risk of bias produced larger effect sizes on the psychological well-being outcome (d=0.57, 95% CI 0.35-0.78) than studies with a high risk of bias (d=0.23, 95% CI 0.10-0.36). The groups were significantly different from each other (χ21=6.9, P=.009). No significant difference was found between the two groups on the work effectiveness outcome.

    Subgroup Analyses

    Subgroup analyses for both the psychological well-being and work effectiveness outcomes were conducted looking at (1) therapeutic approach (CBT vs other), (2) guidance (guided vs nonguided), and (3) population (targeted vs universal). Table 2 shows the results of the subgroup analyses.

    Therapeutic Approach

    Subgroup analysis of the psychological well-being outcome comparing studies using CBT (k=12; as described by the authors) with studies using other psychological approaches (k=9) showed that the groups were not significantly different from each other (χ21=3.63, P=.06), suggesting that for the psychological well-being outcome, the psychological approach used was not a source of heterogeneity. The pooled effect size for studies using the psychological approach of CBT was small (g=0.25, 95% CI 0.10-0.40), and for studies using other psychological approaches it was medium (g=0.52, 95% CI 0.28-0.76). Both are significant effect sizes (z11=3.35, P ≤.001; and z8=4.28, P ≤.001, respectively).

    Subgroup analysis of the work effectiveness outcome comparing studies using predominantly CBT (k=8) with studies using other psychological approaches (k=5) showed that the groups were not significantly different from each other (χ21=0.01, P=.94), suggesting that for the work effectiveness outcome, the therapeutic approach was not a source of heterogeneity. The small pooled effect size for studies using CBT (g=0.26, 95% CI 0.01-0.50) and other psychological approaches (g=0.25, 95% CI 0.11-0.39) are significant (z7=2.05, P=.04; and z4=3.47, P≤.001, respectively).

    Guidance

    Subgroup analysis of the psychological well-being outcome comparing interventions providing guidance (k=10) with interventions that were self-guided (k=11) showed that the groups were not significantly different from each other (χ21=0.11, P=.74), suggesting that for the psychological well-being outcome, guidance was not a source of heterogeneity. The pooled effect size for both groups was small (guided interventions: g=0.39, 95% CI 0.18-0.61; and self-guided interventions: g=0.34, 95% CI 0.16-0.53) both were significant effect sizes (z9=3.58, P≤.001; and z10=3.63, P≤.001, respectively).

    Subgroup analysis of the work effectiveness outcome comparing interventions providing guidance (k=7) with interventions that were self-guided (k=6) showed that the groups were not significantly different from each other suggesting that for the work effectiveness measure, guidance was not a source of heterogeneity (χ21=0.1, P=.81). The pooled effect size for studies using interventions that are guided was a small significant effect size (g=0.27, 95% CI 0.08-0.45; z6=2.84, P=.005). The pooled effect size for interventions that are self-guided was a small nonsignificant effect size (g=0.23, 95% CI −0.06 to 0.51; z5=1.55, P=.12).

    Table 2. Results of subgroup analyses investigating the effect of therapeutic approach, guidance, and population on outcome.
    View this table
    Population

    Subgroup analysis of the psychological well-being outcome comparing a targeted working population (k=9) with a universal working population (k=12) showed that the groups were not significantly different from each other (χ21= 3.59, P=.06), suggesting that for the psychological well-being outcome, population was not a source of heterogeneity. The pooled effect size for the targeted working population was medium (g=0.52, 95% CI 0.28-0.75) and for the universal working population it was small (g=0.25, 95% CI 0.11-0.40). Both were significant effect sizes (z8=4.32, P ≤.001 and z11=3.39, P ≤.001, respectively).

    Subgroup analysis of the work effectiveness outcome comparing a targeted working population (k=7) with a universal working population (k=6) showed that the groups were not significantly different from each other (χ21=0.81, P=.37), suggesting that for the work effectiveness measure, population was not a source of heterogeneity. The pooled effect size for both groups was small (targeted working population: g=0.32, 95% CI 0.04-0.61, and universal working population: g=0.18, 95% CI 0.06-0.30); both effect sizes were significant (z6=2.21, P=.03, and z5=3.00, P=.003, respectively).

    Positive Deviance Analysis

    Of the 21 studies included in this review, 6 studies were in the 70th percentile and above for the lowest attrition in the intervention group [41,47,48,51,57,60], and 4 studies were in the 70th percentile and above for the highest intervention completion [48,49,51,57]. Three studies appeared in both groups [48,51,57], leaving 7 unique studies [41,47-49,51,57,60] that we reviewed in depth to generate hypotheses about intervention features associated with the highest levels of engagement.

    The mean percentage of intervention group attrition in the high engagement group was 8% (SD 4.4), and for the other studies it was 31% (SD 14.5). The mean of the highest intervention completion reported by the authors for the high engagement group was 68% (SD 22.0) and for the other studies it was 33% (SD 26.0).

    Interventions presented in the 7 studies in the high engagement group were reviewed. The interventions for 5 out of the 7 studies offered guidance (71%), compared with only 5/14 of the remaining studies (36%). The mean number of weeks that the intervention was delivered in the high engagement group was 6.6 (SD=0.54, range 6-7 weeks), compared with a mean of 8.1 (SD=3.0, range 4.3-13.0 weeks) in the other studies. All 7 of the studies in the high engagement group described the use of persuasive technology (5/7, 71% self-monitoring, 4/7, 57% tailoring, 1/7, 14% tunneling), compared with 10/14 (71%) in the remaining studies (8/14, 57% tailoring, 5/14, 29% self-monitoring, and 2/14, 14% tunneling).

    Six of the 7 studies (86%) in the high engagement group utilized a secondary modality for delivering the intervention and engaging users (4 studies used emailing and 3 studies used texting), compared with only 8 of the remaining 14 studies (57%). Only 2 of the 21 studies included in this review used a mobile phone app as their primary modality for delivering the intervention; both studies were included in the high engagement group.

    Hypotheses Generation

    These findings suggest that interventions that achieve the greatest engagement and adherence offer guidance, are delivered over a shorter time frame (6 to 7 weeks), utilize secondary modalities for delivering the intervention and engaging users (ie, email and text messages), and use persuasive technology (ie, self-monitoring and tailoring). There is also a suggestion that a mobile phone app is a promising modality for engaging users of occupational digital mental health interventions.


    Discussion

    This review is the first meta-analysis that brings together RCTs of occupational digital mental health interventions and allows us to draw conclusions about both psychological well-being and work effectiveness outcomes. The adaptation of the positive deviance approach was helpful in enabling us to identify and explore in depth the features of high performing interventions in order to generate hypotheses about the intervention features that may promote engagement.

    Study Characteristics

    The 21 studies included in this review recruited and randomized 5260 participants. They were predominantly recruited from the knowledge sector (ie, communication, finance, business, information, research, and education services). The mean reported completion of interventions was 45%. These rates are similar to adherence rates reported for digital health (50%) [62] and digital CBT (median 56%) [63] interventions and are slightly less than those reported for guided digital CBT interventions (67.5%) [64]. Mean study attrition was higher for the intervention groups (23%) than for the control groups (13%). This is in line with a review of computerized CBT [63], which reported that participants in the intervention arm were twice as likely to drop out.

    Intervention Characteristics

    Over half of the studies included in this review used interventions that were predominantly based on CBT (57%). The mean duration of the interventions was 7.6 weeks, with just under half (48%) of the interventions offering some form of guidance. The mean adherence to the interventions was 45%. In a review of digital health interventions, Kelders et al [62] reported a mean duration of 10 weeks, adherence of 50%, and 76% of interventions offering some form of guidance, suggesting that occupational digital mental health interventions may differ somewhat from broader digital health interventions.

    In this review, 81% of the interventions described in the studies used some form of persuasive technology: tailoring was used by 57%, self-monitoring by 43%, and tunneling by 14%. Kelders et al [62] report that for the 48 mental health studies that were included in their review of digital health interventions, tailoring was used by 90%, self-monitoring by 12%, and tunneling by 100%. The discrepancy between the number and type of persuasive technologies identified in our review and the Kelders et al [62] review is explained by differences in coding. For example, Kelders et al [62] did not code computer-mediated communication as persuasive technology, whereas we did. If a coach provided personalized feedback on assignments, we coded this as tailoring, whereas Kelders et al [62] only coded technology initiated communication (ie, when an automated message was sent). This and other differences in the coding make a comparison between the two reviews difficult.

    Meta-Analyses Findings

    Our results indicate that digital mental health interventions delivered in the workplace produced a small positive effect on psychological well-being (g=0.37, 95% CI 0.23-0.50, k=21), and a small positive effect on work effectiveness (g=0.25, 95% CI 0.09-0.41, k=13).

    Our findings situate occupational digital mental health interventions as comparable with other (nondigital specific) occupational interventions in terms of impact on mental health and work effectiveness. The psychological well-being effect size is smaller but not significantly different from the medium effect size reported for a meta-analysis of occupational stress management interventions (d=0.53 95% CI 0.36-0.69) [65] and is larger but not significantly different from the small effect sizes reported in meta-analyses of occupational resilience building programs (d=0.21, 95% CI 0.13-0.29) [66] and health promotion in the workplace programs (depression: g=0.28, 95% CI 0.12-0.44; anxiety g=0.29, 95% CI 0.06-0.53) [67], suggesting that on the psychological outcome, digital mental health interventions have a comparable effect with other occupational interventions. The work effectiveness effect size is comparable with the small effect size reported in a meta-analysis of work engagement interventions (g=0.29, 95% CI 0.12-0.46) [68], suggesting that digital mental health interventions have comparable effects with alternative approaches to enhancing engagement in the workplace.

    The psychological well-being effect size for occupational digital mental health interventions in our review is also comparable with digital mental health interventions delivered in health and community settings for adults with depression [17,22] and similar to digital stress management interventions delivered in community, occupational, and health contexts [24]. Eight studies in the Heber et al [24] review also met the criteria for inclusion in the present review, but less than half of the 23 studies were set within an occupational context.

    Our findings suggest that occupational digital mental health interventions are as effective at improving mental health outcomes as are other more traditional, nondigital occupational programs and other digital interventions delivered in nonoccupational settings. This is impressive given that the workplace context may impact on the way that digital mental health interventions are delivered and received. For example, it has been suggested that two of the advantages of digital health interventions compared with face-to-face or group interventions are increased accessibility, with participants being able to access at a time and a pace convenient for them [23,25,27,30,69], and increased anonymity [23,27,30]. It is these perceived advantages that researchers suggest make digital interventions particularly suited to the workplace [70]. But it is possible that these attributes don’t manifest as advantages in occupational settings; the lack of structure around “attending” digital health interventions may impact on uptake and attendance. Face-to-face or group interventions have a predetermined time for attendance during the working day, possibly with monitoring or reporting of participation to line managers. Digital mental health interventions tend to have less formal attendance with participants expected to attend at a time convenient to them. This flexibility and lack of monitoring, especially among a stressed population who may perceive themselves as time poor, may have a negative impact on intervention engagement; participants may not prioritize the time they need to engage with the intervention during their working day and may resent the intrusion of what they could perceive as work into their evening or weekends.

    Furthermore, within an occupational setting, accessing digital mental health interventions may not be anonymous or even confidential. Access to the intervention may be managed through line management or occupational health; employees that do not have job autonomy may need to get permission to access the intervention during the working day, and employees working in an open plan office or sharing computer equipment may feel exposed when accessing the intervention at work. It is also possible that during the working day employees are so invested in appearing competent and strong that they are not willing or able to engage with a digital mental health intervention. The workplace may not be the appropriate setting to embrace the vulnerability that comes with acknowledging and addressing mental health challenges.

    Further research is needed to gain a clearer understanding of the challenges and benefits of delivering digital mental health interventions within occupational settings. Nevertheless, despite the possibility that the workplace may provide additional challenges to the way that these interventions are delivered and received, our study has shown that occupational digital mental health interventions are effective at improving psychological well-being and work effectiveness.

    Subgroup Analyses

    Therapeutic Approach

    The results of our review would suggest that as it is currently being delivered; CBT-based occupational digital mental health interventions are not producing superior results compared with digital interventions using other psychological approaches. Subgroup analysis comparing studies in our review using approaches described by the study authors as cognitive or cognitive behavioral therapy with studies using other psychological approaches revealed that the groups are not significantly different from each other on either the psychological well-being or work effectiveness measures. These findings are contrary to the established literature.

    A meta-analysis of digital psychological treatments for adult depression also found no difference between CBT and other approaches [17], but a meta-analysis of digital psychological interventions for a range of problems did report a larger effect size for interventions using CBT compared with other therapeutic approaches [71]. Furthermore, meta-analyses on digital CBT consistently report higher effect sizes than were found in this review [20,23].

    One explanation for this may be that as they are currently being delivered, CBT-based digital mental health interventions are not optimized for delivery in occupational settings. In a recent review of occupational digital health, Lehr et al [16] observed that the theoretical background for many of these predominantly CBT-based interventions fails to incorporate theoretical frameworks of occupational stress. Relevant theoretical models include the effort reward imbalance model [72], the person-environment fit model (for an overview see [73]), and the job demands-control model [74]. Incorporating these frameworks into the content of occupational digital mental health interventions may make the interventions more relevant and sensitive to the workplace [16] and may increase the capacity of all psychological approaches to meet the needs of occupational groups.

    Guidance

    No significant difference was found in our review between interventions that provide guidance with those that are self-guided. This is different to the established literature, which has consistently found that guided Internet interventions are significantly superior to unguided interventions [17,22-24,28,75-78]. A review by Grist and Cavanagh [20] on computerized CBT for common mental health problems also found no significant difference in effect size between guided and unguided programs. The authors suggested caution in interpreting their findings as only 5 studies using unguided programs had been identified. Low power from a small number of studies may also be an issue for this study; consequently, we too suggest caution in interpreting these findings. Another explanation for these findings may be the failure of this review to adequately code and differentiate the extent and form of guidance that is offered to participants and the extent to which that guidance is utilized. A recent review of digital interventions for stress differentiated between (1) guided interventions, (2) adherence-focused guidance (feedback on request), and (3) unguided interventions that provided email or telephone reminders [24]. This review did not make such a distinction, differentiating solely between interventions that did not describe guidance in any form and interventions that did describe some form of guidance. Furthermore, some studies’ failure to adequately describe the in-program-support offered to participants may have resulted in some studies being wrongly categorized as unguided or guided. It is also unclear from some of the study descriptions whether support was being offered to participants outside the digital intervention, such as from an employee assistance program (EAP) or an occupational health team.

    It is worth noting that the positive deviance analysis found 71% of studies in the high engagement group offered guidance compared with only 36% in the remaining studies, suggesting that there may be a link between the provision of guidance and increased engagement with occupational digital mental health interventions.

    Targeted and Universal Populations

    No significant differences were found in the review between studies that recruited a targeted population (elevated levels of depression, stress, and insomnia) and studies that targeted a universal population for either well-being or work effectiveness outcomes. However, there was a trend in both cases for studies with a targeted population to have a larger effect size, suggesting that individuals with raised levels of stress, depression, and insomnia benefit more from occupational digital mental health. One explanation for this might be that the measures used may not be sensitive to change at the lower end of the scale. Another explanation might be that participants with raised levels of psychological distress may be more motivated to implement the learning in the program and therefore produce more immediate post intervention results.

    These findings are contrary to a meta-analysis on workplace resilience interventions, which found weaker effects among targeted populations compared with universal populations at post intervention [66]. That study reported that the effects of occupational resilience-building diminished sharply over time among the universal population but increased in the targeted population, suggesting that for a resilience-building program the benefits amongst a targeted population may increase with time [66].

    Positive Deviance

    Maximizing engagement with, and adherence to, digital heath interventions remains a pressing concern. The partial implementation of the positive deviance approach used in this review suggests that, within an occupational setting, These findings suggest that interventions that achieve the greatest engagement and adherence offer guidance, are delivered over a shorter time frame (6 to 7 weeks), utilize secondary modalities for delivering the intervention and engaging users (ie, email and text messages), and use persuasive technology (ie, self-monitoring and tailoring). These findings echo the literature on digital health interventions. In reviews of the design features that promote adherence to digital health interventions, evidence has also been found for increased guidance [62], the shorter duration of the intervention [78], contact through email or phone [30], and incorporating tailoring and self-monitoring [79]. Meta-analyses of occupational stress management interventions [65], digital stress management in the general adult population [24], and digital psychological treatment for depression [22] also found evidence for the increased effectiveness of interventions delivered over a similar period. We would recommend the development and testing of optimized occupational digital mental health interventions based on these principles.

    Only 2 of the 21 studies included in this review used a mobile phone app as their primary modality of intervention delivery. Both studies were included in the high engagement group, suggesting that app technology is a promising modality for engaging users of occupational digital mental health interventions.

    Limitations

    This study highlights limitations in the broader digital mental well-being literature. One limitation is the small number of studies that measured occupational outcomes. Although the studies included in the review were aimed at employed participants and delivered within workplace contexts, most of them reported the reduction of psychological symptoms and failed to report occupational outcomes. We would recommend that future trials of psychological interventions delivered in the workplace incorporate occupational outcome measures, including work effectiveness.

    Another limitation was the considerable heterogeneity that was found across the studies. This included variation in the measures used (particularly in the work effectiveness measures), variations in the guidance given and the adherence, therapeutic approach and delivery of interventions, variation in the participants including country, type of organization, role and symptom severity, and variation in the quality of the study. The large number of unclear and high-risk of bias ratings limit the quality of the studies included in the review. The variation across the studies suggests that the results of our study should be interpreted with caution. We recommend that future research uses more robust study designs.

    The coding used in the review was limited by the description given about the interventions in the published literature. Many of the descriptions were short and appeared incomplete. This is a limitation described by other researchers [30,62,80]. Naturally, incomplete descriptions, especially descriptions of the persuasive technology and guidance, limit the strength of the conclusions that can be drawn here.

    Other limitations specific to this review include the use in the positive deviance analysis of intervention completion and intervention group attrition as proxy measures of intervention engagement and adherence; the number of times that a participant logs in to an intervention or the number of modules that they complete cannot necessarily be taken as a measure of the extent to which they engage psychologically with the intervention [80,81]; Likewise, the extent to which participants comply with the study protocol is not a perfect measure of psychological engagement. It is reassuring to note, however, that a review of adherence and its impact on digital therapies [82] reported that module completion was found to be the adherence measure most related to outcomes in psychological health interventions. Other limitations to the review include our use of the term “psychological well-being.” We recognize that psychological well-being is more than the absence of stress or depression and that our use of the term in this review does not capture aspects of well-being such as autonomy, personal growth, functioning, and relationships with others. Finally, this review did not analyze follow-up data, so we are unable to draw conclusions on the long-term effect of digital occupational mental health programs.

    Implications

    This review has demonstrated that delivering digital mental health interventions in the workplace can result in improved psychological well-being and work effectiveness. Our findings suggest that interventions that achieve the greatest engagement and adherence offer guidance, are delivered over a shorter time frame (6 to 7 weeks), utilize secondary modalities for delivering the intervention and engaging users (ie, email and text messages), and use persuasive technology (ie, self-monitoring and tailoring). Further research is needed to test these hypotheses.

    We recommend that researchers and developers of occupational digital mental health interventions acknowledge the importance of the workplace setting in the content, delivery, and analysis of their interventions. We strongly recommend that therapeutic approaches incorporate relevant theoretical frameworks of occupational stress and that further research is conducted to better understand the challenges and benefits to delivering digital mental health interventions in the workplace. We also recommend that researchers incorporate in future research nonclinical measures of psychological distress and measures of occupational outcomes so that we can learn more about the psychological and occupational impact of digital mental health. A future area of research would be the long-term effect of these interventions.

    Conclusions

    This review provides evidence that occupational digital mental health interventions can improve workers’ psychological well-being and increase work effectiveness and identifies intervention characteristics that may increase engagement. We recommend that researchers and intervention developers recognize that the workplace is a dynamic and complex environment that may affect the way that individuals receive and engage with digital mental health interventions.

    Acknowledgments

    The authors would like to thank Kim Aikens, Douglas W Billings, Amelia Birney, Royer F Cook, Dan Hasson, Elena Heber, and Rino Umanodan for providing further information on their studies.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Selected characteristics of included studies.

    PDF File (Adobe PDF File), 142KB

    Multimedia Appendix 2

    Selected characteristics of included interventions.

    PDF File (Adobe PDF File), 147KB

    References

    1. Leka S, Jain A. Health impact of psychosocial hazards at work: An overview. Geneva: World Health Organisation; 2010.
    2. Harris Interactive. American Psychological Association. 2011 Mar. Stress in the Workplace: Survey Summary   URL: https://www.apa.org/news/press/releases/phwa-survey-summary.pdf [accessed 2017-06-27] [WebCite Cache]
    3. Birnbaum HG, Kessler RC, Kelley D, Ben-Hamadi R, Joish VN, Greenberg PE. Employer burden of mild, moderate, and severe major depressive disorder: mental health services utilization and costs, and work performance. Depress Anxiety 2010;27(1):78-89. [CrossRef] [Medline]
    4. Dewa CS, McDaid D, Ettner SL. An international perspective on worker mental health problems: who bears the burden and how are costs addressed? Can J Psychiatry 2007 Jun;52(6):346-356. [CrossRef] [Medline]
    5. Dewa CS, Hoch JS. Barriers to mental health service use among workers with depression and work productivity. J Occup Environ Med 2015 Jul;57(7):726-731 [FREE Full text] [CrossRef] [Medline]
    6. Martin JK, Blum TC, Beach SR, Roman PM. Subclinical depression and performance at work. Soc Psychiatry Psychiatr Epidemiol 1996 Jan;31(1):3-9. [Medline]
    7. Sanderson K, Andrews G. Common mental disorders in the workforce: recent findings from descriptive and social epidemiology. Can J Psychiatry 2006 Feb;51(2):63-75. [CrossRef] [Medline]
    8. Corbière M, Shen J, Rouleau M, Dewa CS. A systematic review of preventive interventions regarding mental health issues in organizations. Work 2009;33(1):81-116. [CrossRef] [Medline]
    9. Lexis MA, Jansen NW, Huibers MJ, van Amelsvoort LG, Berkouwer A, Tjin AT, et al. Prevention of long-term sickness absence and major depression in high-risk employees: a randomised controlled trial. Occup Environ Med 2011 Jun;68(6):400-407. [CrossRef] [Medline]
    10. Dewa CS, Thompson AH, Jacobs P. The association of treatment of depressive episodes and work productivity. Can J Psychiatry 2011 Dec;56(12):743-750. [CrossRef] [Medline]
    11. Wang PS, Simon GE, Avorn J, Azocar F, Ludman EJ, McCulloch J, et al. Telephone screening, outreach, and care management for depressed workers and impact on clinical and work productivity outcomes: a randomized controlled trial. J Am Med Assoc 2007 Sep 26;298(12):1401-1411 [FREE Full text] [CrossRef] [Medline]
    12. Mykletun A, Harvey SB. Prevention of mental disorders: a new era for workplace mental health. Occup Environ Med 2012 Dec;69(12):868-869. [CrossRef] [Medline]
    13. Lim D, Sanderson K, Andrews G. Lost productivity among full-time workers with mental disorders. J Ment Health Policy Econ 2000 Sep 01;3(3):139-146. [Medline]
    14. Tustin N. The role of patient satisfaction in online health information seeking. J Health Commun 2010 Jan;15(1):3-17. [CrossRef] [Medline]
    15. Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth). Annu Rev Clin Psychol 2007;3:53-76. [CrossRef] [Medline]
    16. Lehr D, Geraedts A, Persson Asplund R, Khadjesari Z, Heber E, de Bloom J, et al. Occupational e-Mental Health: Current Approaches and Promising Perspectives for Promoting Mental Health in Workers. In: Wiencke M, Cacace M, Fischer S, editors. Health at Work: Interdisciplinary Perspectives. Switzerland: Springer International Publishing; 2016:257-281.
    17. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205. [CrossRef] [Medline]
    18. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
    19. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther 2009;38(2):66-82. [CrossRef] [Medline]
    20. Grist R, Cavanagh K. Computerised cognitive behavioural therapy for common mental health disorders, what works, for whom under what circumstances? A systematic review and meta-analysis. J Contemp Psychother 2013 Sep 4;43(4):243-251 [FREE Full text] [CrossRef]
    21. Mewton L, Smith J, Rossouw P, Andrews G. Current perspectives on Internet-delivered cognitive behavioral therapy for adults with anxiety and related disorders. Psychol Res Behav Manag 2014;7:37-46 [FREE Full text] [CrossRef] [Medline]
    22. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    23. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
    24. Heber E, Ebert DD, Lehr D, Cuijpers P, Berking M, Nobis S, et al. The benefit of Web and computer-based interventions for stress a systematic review and meta-analysis. J Med Internet Res 2017 Feb 17;19(2):e32 [FREE Full text] [CrossRef] [Medline]
    25. Cavanagh K, Millings A. Increasing engagement with computerised cognitive behavioural therapies. ICST Trans Amb Sys 2013 Feb 22;13(2):e3 [FREE Full text] [CrossRef]
    26. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    27. Kohl LF, Crutzen R, de Vries NK. Online prevention aimed at lifestyle behaviors: a systematic review of reviews. J Med Internet Res 2013;15(7):e146 [FREE Full text] [CrossRef] [Medline]
    28. Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther 2012 Aug;50(7-8):463-468. [CrossRef] [Medline]
    29. Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health 2009;30:273-292. [CrossRef] [Medline]
    30. Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011;13(1):e2 [FREE Full text] [CrossRef] [Medline]
    31. Norman GJ, Zabinski MF, Adams MA, Rosenberg DE, Yaroch AL, Atienza AA. A review of eHealth interventions for physical activity and dietary behavior change. Am J Prev Med 2007 Oct;33(4):336-345 [FREE Full text] [CrossRef] [Medline]
    32. Bradley EH, Curry LA, Ramanadhan S, Rowe L, Nembhard IM, Krumholz HM. Research in action: using positive deviance to improve quality of health care. Implement Sci 2009 May 08;4:25 [FREE Full text] [CrossRef] [Medline]
    33. Marsh DR, Schroeder DG, Dearden KA, Sternin J, Sternin M. The power of positive deviance. Br Med J 2004 Nov 13;329(7475):1177-1179 [FREE Full text] [CrossRef] [Medline]
    34. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. United Knigdom: The Cochrane Collaboration; 2011.
    35. Cohen J. Statistical power analysis for the behavioral sciences (2nd ed). New York: Academic Press; 1988.
    36. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Br Med J 2003 Sep 6;327(7414):557-560 [FREE Full text] [CrossRef] [Medline]
    37. Field A. The problems in using fixed-effects models of meta-analysis on real-world data. Understanding Statistics 2003 Apr 03;2(2):105-124 [FREE Full text] [CrossRef]
    38. Schubart JR, Stuckey HL, Ganeshamoorthy A, Sciamanna CN. Chronic health conditions and internet behavioral interventions: a review of factors to enhance user engagement. Comput Inform Nurs 2011 Feb;29(2):81-92. [CrossRef] [Medline]
    39. Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B. E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther 2007;36(3):179-192. [CrossRef] [Medline]
    40. Yamagishi M, Kobayashi T, Nakamura Y. Effects of web-based career identity training for stress management among Japanese nurses: a randomized control trial. J Occup Health 2008;50(2):191-193 [FREE Full text] [Medline]
    41. Birney AJ, Gunn R, Russell JK, Ary DV. MoodHacker mobile web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth 2016 Jan 26;4(1):e8 [FREE Full text] [CrossRef] [Medline]
    42. Allexandre D, Bernstein AM, Walker E, Hunter J, Roizen MF, Morledge TJ. A web-based mindfulness stress management program in a corporate call center: a randomized clinical trial to evaluate the added benefit of onsite group support. J Occup Environ Med 2016 Mar;58(3):254-264 [FREE Full text] [CrossRef] [Medline]
    43. Billings DW, Cook RF, Hendrickson A, Dove DC. A web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med 2008 Aug;50(8):960-968. [CrossRef] [Medline]
    44. Aikens KA, Astin J, Pelletier KR, Levanovich K, Baase CM, Park YY, et al. Mindfulness goes to work: impact of an online workplace intervention. J Occup Environ Med 2014 Jul;56(7):721-731. [CrossRef] [Medline]
    45. Cook RF, Billings DW, Hersch RK, Back AS, Hendrickson A. A field test of a web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity: randomized controlled trial. J Med Internet Res 2007;9(2):e17 [FREE Full text] [CrossRef] [Medline]
    46. Beauchamp N, Irvine AB, Seeley J, Johnson B. Worksite-based internet multimedia program for family caregivers of persons with dementia. Gerontologist 2005 Dec;45(6):793-801. [Medline]
    47. Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, et al. Self-guided internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med 2016 May;73(5):315-323. [CrossRef] [Medline]
    48. Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res 2016;18(1):e21 [FREE Full text] [CrossRef] [Medline]
    49. Ebert DD, Lehr D, Boß L, Riper H, Cuijpers P, Andersson G, et al. Efficacy of an internet-based problem-solving training for teachers: results of a randomized controlled trial. Scand J Work Environ Health 2014 Nov;40(6):582-596. [CrossRef] [Medline]
    50. Ebert DD, Berking M, Thiart H, Riper H, Laferton JA, Cuijpers P, et al. Restoring depleted resources: efficacy and mechanisms of change of an internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol 2015 Dec;34 Suppl:1240-1251. [CrossRef] [Medline]
    51. Thiart H, Lehr D, Ebert DD, Berking M, Riper H. Log in and breathe out: internet-based recovery training for sleepless employees with work-related strain - results of a randomized controlled trial. Scand J Work Environ Health 2015 Mar;41(2):164-174. [CrossRef] [Medline]
    52. Feicht T, Wittmann M, Jose G, Mock A, von HE, Esch T. Evaluation of a seven-week web-based happiness training to improve psychological well-being, reduce stress, and enhance mindfulness and flourishing: a randomized controlled occupational health study. Evid Based Complement Alternat Med 2013;2013:676953 [FREE Full text] [CrossRef] [Medline]
    53. Bolier L, Ketelaar SM, Nieuwenhuijsen K, Smeets O, Gärtner FR, Sluiter JK. Workplace mental health promotion online to enhance well-being of nurses and allied health professionals: a cluster-randomized controlled trial. Internet Interv 2014;1(4):196-204. [CrossRef]
    54. Geraedts AS, Kleiboer AM, Wiezer NM, van MW, Cuijpers P. Short-term effects of a web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res 2014 May;16(5):e121 [FREE Full text] [CrossRef] [Medline]
    55. Grime PR. Computerized cognitive behavioural therapy at work: a randomized controlled trial in employees with recent stress-related absenteeism. Occup Med (Lond) 2004 Aug;54(5):353-359 [FREE Full text] [CrossRef] [Medline]
    56. Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med 2014 Mar;44(4):741-752 [FREE Full text] [CrossRef] [Medline]
    57. Umanodan R, Shimazu A, Minami M, Kawakami N. Effects of computer-based stress management training on psychological well-being and work performance in japanese employees: a cluster randomized controlled trial. Ind Health 2014;52(6):480-491 [FREE Full text] [Medline]
    58. Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Effects of an Internet-based cognitive behavioral therapy (iCBT) program in Manga format on improving subthreshold depressive symptoms among healthy workers: a randomized controlled trial. PLoS One 2014;9(5):e97167 [FREE Full text] [CrossRef] [Medline]
    59. Abbott JA, Klein B, Hamilton C, Rosenthal A. The impact of online resilience training for sales managers on wellbeing and work performance. EJAP 2009;5(1):89-95. [CrossRef]
    60. Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: a randomized controlled trial. Internet Interv 2014 Jul;1(3):95-101 [FREE Full text] [CrossRef]
    61. Fogg BJ. Persuasive Technology: Using Computers to Change What We Think and Do. San Francisco: Morgan Kaufmann; 2003.
    62. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012;14(6):e152 [FREE Full text] [CrossRef] [Medline]
    63. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [CrossRef] [Medline]
    64. van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One 2014;9(7):e100674 [FREE Full text] [CrossRef] [Medline]
    65. Richardson KM, Rothstein HR. Effects of occupational stress management intervention programs: a meta-analysis. J Occup Health Psychol 2008 Jan;13(1):69-93. [CrossRef] [Medline]
    66. Vanhove A, Herian MN, Perez AL, Harms PD, Lester PB. Can resilience be developed at work? A meta-analytic review of resilience-building programme effectiveness. J Occup Organ Psychol 2015 Apr 25;89(2):278-307 [FREE Full text] [CrossRef]
    67. Martin A, Sanderson K, Cocker F. Meta-analysis of the effects of health promotion intervention in the workplace on depression and anxiety symptoms. Scand J Work Environ Health 2009 Jan;35(1):7-18 [FREE Full text] [Medline]
    68. Knight C, Patterson M, Dawson J. Building work engagement: a systematic review and meta-analysis investigating the effectiveness of work engagement interventions. J Organiz Behav 2016 Dec 13;38(6):792-812 [FREE Full text] [CrossRef]
    69. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res 2006;8(2):e10 [FREE Full text] [CrossRef] [Medline]
    70. Junge M, Lehr D, Bockting C, Berking M, Riper H, Cuijpers P, et al. For whom are internet-based occupational mental health interventions effective? Moderators of internet-based problem-solving training outcome. Internet Interv 2015 Mar;2(1):39-47 [FREE Full text] [CrossRef]
    71. Barak A, Hen L, Boniel-Nissim M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Technol Hum Serv 2008 Jul 03;26(2-4):109-160 [FREE Full text] [CrossRef]
    72. Siegrist J. Adverse health effects of high-effort/low-reward conditions. J Occup Health Psychol 1996 Jan;1(1):27-41. [Medline]
    73. Edwards JR, Caplan RD, Harrison RV. Person-environment fit theory: Conceptual foundations, empirical evidence, and directions for future research. In: Cooper CL, editor. Theories of organizational stress. Oxford: Oxford University Press; 1998:28-67.
    74. Karasek RA. Job demands, job decision latitude and mental strain: implications for job redesign. Adm Sci Q 1979;24:285-308.
    75. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions — A systematic review. Internet Interv 2014 Oct;1(4):205-215 [FREE Full text] [CrossRef]
    76. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    77. Zarski A, Lehr D, Berking M, Riper H, Cuijpers P, Ebert DD. Adherence to internet-based mobile-supported stress management: a pooled analysis of individual participant data from three randomized controlled trials. J Med Internet Res 2016;18(6):e146 [FREE Full text] [CrossRef] [Medline]
    78. Cugelman B, Thelwall M, Dawes P. Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors. J Med Internet Res 2011;13(1):e17 [FREE Full text] [CrossRef] [Medline]
    79. Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012 Mar;18(2):137-144. [CrossRef] [Medline]
    80. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):833-842. [CrossRef] [Medline]
    81. O'Brien HL, Toms EG. The development and evaluation of a survey to measure user engagement. J Am Soc Inf Sci 2009 Oct 19;61(1):50-69 [FREE Full text] [CrossRef]
    82. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    CBT: cognitive behavioral therapy
    EAP: employee assistance program
    RCT: randomized controlled trial
    SD: standard deviation
    WLC: wait list control


    Edited by G Eysenbach; submitted 24.02.17; peer-reviewed by S Kraft, L Tetrick; comments to author 09.05.17; revised version received 22.05.17; accepted 23.05.17; published 26.07.17

    ©Stephany Carolan, Peter R Harris, Kate Cavanagh. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 26.07.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.