This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Nonadherence to treatment is a prevalent issue in Internet interventions. Guidance from health care professionals has been found to increase treatment adherence rates in Internet interventions for a range of physical and mental disorders. Evaluating different guidance formats of varying intensity is important, particularly with respect to improvement of effectiveness and cost-effectiveness. Identifying predictors of nonadherence allows for the opportunity to better adapt Internet interventions to the needs of participants especially at risk for discontinuing treatment.
The goal of this study was to investigate the influence of different guidance formats (content-focused guidance, adherence-focused guidance, and administrative guidance) on adherence and to identify predictors of nonadherence in an Internet-based mobile-supported stress management intervention (ie, GET.ON Stress) for employees.
The data from the groups who received the intervention were pooled from three randomized controlled trials (RCTs) that evaluated the efficacy of the same Internet-based mobile-supported stress management intervention (N=395). The RCTs only differed in terms of the guidance format (content-focused guidance vs waitlist control, adherence-focused guidance vs waitlist control, administrative guidance vs waitlist control). Adherence was defined by the number of completed treatment modules (0-7). An ANOVA was performed to compare the adherence rates from the different guidance formats. Multiple hierarchical linear regression analysis was conducted to evaluate predictors of nonadherence, which included gender, age, education, symptom-related factors, and hope for improvement.
In all, 70.5% (93/132) of the content-focused guidance sample, 68.9% (91/132) of the adherence-focused guidance sample, and 42.0% (55/131) of the participants in the administrative guidance sample completed all treatment modules. Guidance had a significant effect on treatment adherence (
Guidance has been shown to be an influential factor in promoting adherence to an Internet-based mobile-supported stress management intervention. Adherence-focused guidance, which included email reminders and feedback on demand, was equivalent to content-focused guidance with regular feedback while requiring only approximately a quarter of the coaching resources. This could be a promising discovery in terms of cost-effectiveness. However, even after considering guidance, sociodemographic, and symptom-related characteristics, most interindividual differences in nonadherence remain unexplained.
DRKS00004749; http://drks-neu.uniklinik-freiburg.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL _ID=DRKS00004749 (Archived by WebCite at http://www.webcitation.org/6QiDk9Zn8); DRKS00005112; http://drks-neu.uniklinik-freiburg. de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00005112 (Archived by WebCite at http://www.webcitation.org/6QiDysvev); DRKS00005384; http://drks-neu.uniklinik-freiburg.de/ drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00005384 (Archived by WebCite at http://www.webcitation.org/6QiE0xcpE)
Many participants of Internet-based programs do not begin the interventions after registration or are nonadherent and quit the intervention prematurely against recommendations in both study and routine care settings [
Occupational stress is associated with an increased risk for common mental disorders in the long term [
In order to improve the clinical effects of Internet interventions, it is important to identify and evaluate factors associated with adherence [
Guidance in Internet interventions is often classified according to the amount of required coaching or therapist time [
Content-focused guidance in Internet interventions has been found to be associated with higher levels of treatment completion (72%) compared to administrative guidance (62%) or unguided interventions (26%) in a meta-analysis [
However, whether adherence-focused guidance results in comparable adherence rates to content-focused guidance formats remains unclear. Comparisons of adherence-focused guidance to content-focused guidance are currently limited regarding treatment adherence in Internet interventions.
Mohr and colleagues [
Based on the supportive accountability model [
Apart from treatment factors, knowledge of user characteristics related to nonadherence helps to identify individuals who are at risk of discontinuing treatment and might need additional support. According to the behavior change model for Internet interventions by Ritterband and colleagues [
Demographic and disease-specific user characteristics were of particular interest in past adherence research in the field of Internet interventions. Low education level [
High symptom severity at baseline is also frequently linked to lower treatment adherence [
Only a few studies to date have investigated predictors of adherence in Internet intervention and they showed conflicting results [
To the best of our knowledge, no research to date has explored the influence of different guidance formats and user characteristics on treatment adherence to an Internet intervention in stressed employees.
The current study aimed to (1) report adherence rates from a newly developed Internet-based mobile-supported stress management intervention (ie, GET.ON Stress) and (2) investigate the role of different guidance formats (content-focused guidance, adherence-focused guidance, and administrative guidance) on adherence. Further goals of the study were to (3) identify user characteristics predictive of treatment nonadherence over and above the guidance formats and (4) analyze differential predictor effects as a function of guidance formats.
We hypothesized that (1) treatment adherence rates would be greater for content-focused guidance and adherence-focused guidance compared to administrative guidance, and (2) adherence-focused guidance is equivalent to content-focused guidance in terms of adherence rates. User characteristics that contribute to treatment nonadherence apart from guidance formats and differential effects of predictors as a function of guidance formats were analyzed exploratively. In this study, treatment adherence is operationalized by the number of completed treatment modules.
Data for this analysis were drawn from three randomized controlled trials (RCTs) evaluating the same Internet-based mobile-supported stress management intervention (GET.ON Stress [
The analyses in this study were based solely on the intervention group samples of the N=395 participants who received the same Internet-based mobile-supported stress management intervention (study 1: n=132; study 2: n=132; study 3: n=131). Participants in the waitlist control condition were not included in the analyses because they did not receive access to the training until 6 months after randomization. All three studies included (1) currently employed workers, (2) older than age 18 years, (3) with scores ≥22 on the Perceived Stress Scale (PSS-10) [
The Internet-based mobile-supported stress management intervention GET.ON Stress is based on two main components: problem solving [
Participants in the content-focused guidance condition received personalized written feedback from an eCoach on the exercises they had completed in each module within 48 hours. The eCoaches were psychologists and trained Master’s-level psychology students who followed guidelines about the feedback process that were defined according to the standardized manual for the intervention. The eCoaches were advised to not spend more than 30 minutes on feedback on a given completed module. The eCoaches sent reminders when the participants did not complete a module within 7 days. In total, the eCoaches sent 365 reminders, corresponding to a mean 2.77 reminders per participant (range 0-11, SD 2.41). The time required for coaching totaled up to 4 hours per participant.
Participants of the adherence-focused guidance condition were also supported by an eCoach. The guidance manual was based on our developed adherence-focused guidance concept as outlined in the Introduction [
Participants in the administrative guidance condition were provided with contact information for the study administration team during the study period, which addressed such things as the completion of questionnaires, but they were not supported by an eCoach. They were provided with an email address to use in case of any technical problems.
The number of completed treatment modules in the Internet-based mobile-supported stress management intervention, which ranged from 0 to 7, was the primary outcome measure in this study and was assessed by the system that provided the intervention. Module completion was defined by completion of the last page of a module. To arrive on the last page, participants were required to complete all the previous writing tasks. A module completion score of 0 could either mean that the participant did not start the intervention or did not finish the first module. Each module took approximately 45 to 60 minutes for completion.
The following variables, assessed at baseline before the start of the program, were evaluated as potential predictors of nonadherence: sociodemographic factors (gender [male/female], age [years], level of education [low, middle, high]), symptom severity factors (perceived stress, depressive symptoms, emotional exhaustion), and hope for improvement (confidence in treatment efficacy).
Perceived stress at baseline relating to the past week was examined with the German version of the 10-item Perceived Stress Scale (PSS-10) [
Baseline depression symptom severity was measured with the German version of the Center for Epidemiological Studies Depression Scale (CES-D) [
To measure emotional exhaustion, the basic stress dimension of burnout, the German version of the Maslach Burnout Inventory was utilized (MBI-GS-D) [
Hope for improvement (confidence in treatment efficacy) was measured using the homonymous subscale of the German Patient Questionnaire on Therapy Expectation and Evaluation (PATHEV) [
An ANOVA was conducted to compare the treatment adherence rates between the three guidance formats with guidance as the independent variable and adherence as the dependent variable [
In planned contrasts, the superiority of the content-focused guidance and adherence-focused guidance over administrative guidance as well as equivalence of adherence-focused guidance and content-focused guidance was assessed. The effect sizes for the planned comparisons were described by
All continuous predictors were group mean centered. The power analysis revealed that with the given sample size (N=395), small effects (
One participant had missing values in the depression level and the hope of improvement variable for which data were imputed using a Markov chain Monte Carlo multivariate imputation algorithm (missing data module in SPSS version 22) with 100 estimations per missing value.
In order to test the robustness of our results, we applied sensitivity analyses. We used (1) a more conservative outcome by defining modules as completed only when finished within 12 weeks and (2) Kaplan-Meier survival curves to compare the adherence rates between the three different guidance formats. All analyses were performed using SPSS version 22. Directed hypotheses were tested with a one-tailed test and nondirected hypotheses with a two-tailed test.
In total, 395 participants were included in the analysis. Baseline characteristics of the study population are presented in
Baseline characteristics of the study population (N=395).
Characteristic | Administrative guidance (n=131) | Adherence-focused guidance (n=132) | Content-focused guidance (n=132) | ||
Age (years), mean (SD) | 41.2 (9.4) | 42.6 (9.5) | 42.4 (10.7) | .51 | |
.02 | |||||
Female | 97 (74.1) | 113 (85.6) | 97 (73.5) | ||
Male | 34 (26.0) | 18 (13.6) | 35 (26.5) | ||
Other | 1 (0.8) | ||||
.98 | |||||
Caucasian | 107 (81.7) | 108 (81.8) | 110 (83.3) | ||
Asian | 1 (0.8) | 1 (0.8) | 0 | ||
Not reported | 23 (17.6) | 23 (17.4) | 22 (16.7) | ||
.96 | |||||
Unmarried | 40 (30.5) | 39 (29.6) | 43 (32.6) | ||
Married | 65 (49.6) | 62 (47.0) | 63 (47.7) | ||
Cohabited | 16 (12.2) | 18 (13.6) | 17 (12.9) | ||
Separated | 9 (6.9) | 13 (9.9) | 8 (6.1) | ||
Widowed | 1 (0.8) | 0 | 1 (0.8) | ||
.01 | |||||
Low | 0 | 1 (0.8) | 5 (3.8) | ||
Middle | 57 (43.5) | 62 (47.0) | 42 (31.8) | ||
High | 74 (56.5) | 69 (52.3) | 85 (64.4) | ||
.34 | |||||
Low | 39 (29.8) | 41 (31.1) | 35 (26.5) | ||
Middle | 40 (30.5) | 33 (25.0) | 26 (19.7) | ||
High | 45 (34.4) | 49 (37.1) | 59 (44.7) | ||
Not reported | 7 (5.3) | 9 (6.8) | 12 (9.1) | ||
.42 | |||||
Permanent | 104 (79.4) | 107 (81.1) | 110 (83.3) | ||
Temporary | 19 (14.5) | 14 (10.6) | 11 (8.3) | ||
Self-employed | 6 (4.6) | 9 (6.8) | 11 (8.3) | ||
Other | 2 (1.5) | 2 (1.5) | 0 | ||
.82 | |||||
Yes | 14 (10.7) | 17 (12.9) | 17 (12.9) | ||
No | 117 (89.3) | 115 (87.1) | 115 (87.1) | ||
.32 | |||||
Yes | 47 (35.9) | 46 (34.9) | 57 (43.2) | ||
No | 84 (64.1) | 86 (65.2) | 75 (56.8) | ||
Stress, mean (SD) | 25.7 (5.0) | 25.2 (4.6) | 25.9 (3.9) | .44 | |
Depression, mean (SD) | 25.1a (9.3) | 23.2 (9.3) | 23.3 (8.5) | .15 | |
Emotional exhaustion, mean (SD) | 4.8 (0.8) | 4.7 (0.8) | 4.7 (0.7) | .96 | |
Hope of improvement, mean (SD) | 3.7a (0.6) | 3.6 (0.6) | 3.7 (0.7) | .27 |
a Due to missing data, the means refer to a subsample with n=130 in this group.
Total number of completed modules per participant.
Number of completed modules by module.
As expected, there was a significant effect of guidance on treatment adherence (
Using a more conservative outcome by defining modules as completed only when finished within 12 weeks did not result in any different conclusions (results not shown). Similarly, conducting survival analysis according to Kaplan-Meier, we derived comparable results. The survival distribution for administrative guidance was significantly different from the survival distribution of adherence-focused guidance (χ22=19.0,
Results of the imputed hierarchical multiple linear regression analysis to identify user characteristics predictive of treatment nonadherence over guidance formats.
Variable | B (SE)b | Betac | |||
.02 | .09 | ||||
Constant | 3.22 (1.08) | ||||
Gender | 0.53 (0.31) | .09 | .09 | ||
Education | 0.47 (0.25) | .10 | .06 | ||
Age | –0.00 (0.01) | –.01 | .79 | ||
.02 | .046 | ||||
Constant | 0.28 (1.56) | .86 | |||
Gender | 0.42 (0.31) | .07 | .17 | ||
Education | 0.44 (0.24) | .09 | .07 | ||
Age | –0.01 (0.01) | –.03 | .58 | ||
Stress | 0.08 (0.04) | .14 | .046 | ||
Depression | –0.04 (0.02) | –.15 | .04 | ||
Emotional exhaustion | 0.31 (0.21) | .09 | .14 | ||
Hope for improvement | 0.26 (0.20) | .07 | .20 | ||
.05 | <.001 | ||||
Constant | –0.49 (1.53) | ||||
Gender | 0.32 (0.30) | .05 | .29 | ||
Education | 0.44 (0.24) | .09 | .06 | ||
Age | –0.01 (0.01) | –.04 | .39 | ||
Stress | 0.07 (0.04) | .12 | .08 | ||
Depression | –0.03 (0.02) | –.11 | .13 | ||
Emotional exhaustion | 0.28 (0.20) | .08 | .16 | ||
Hope for improvement | 0.36 (0.20) | .09 | .07 | ||
Guidance format (content-focused guidance vs administrative guidance) | 1.31 (0.31) | .24 | <.001 | ||
Guidance format (adherence-focused guidance vs administrative guidance) | 1.24 (0.31) | .23 | <.001 |
a
bUnstandardized regression coefficient and unstandardized standard error.
cStandardized regression coefficient.
Adding the interactions between the guidance formats and the other variables to the model did not significantly change explained variance (∆
The first aim of this study was to identify the adherence rates for an Internet-based mobile-supported stress management intervention. The content-focused guidance adherence rate was 71%, which is comparable to the rates found in other guided Internet-based stress management interventions (46%-88%) [
Our second research goal was to investigate the influence of different guidance formats on adherence in an Internet-based mobile-supported stress management intervention. Similarly to studies on other target conditions, such as depression [
As hypothesized, both content-focused guidance and adherence-focused guidance have high adherence rates; therefore, the next step was to analyze their equivalence in terms of treatment adherence. For both guidance formats, adherence was equivalent. Despite the equivalence in adherence rates, both guidance formats differ in the amount of eCoaching each requires. Content-focused guidance included both reminders and written feedback from an eCoach on every completed module and required up to 4 hours of coaching time. In contrast, adherence-focused guidance consisted of adherence monitoring and feedback on demand and only required up to 1 hour of coaching time per participant during the intervention. Therefore, by choosing adherence-focused guidance regarding the costs of treatment, substantial savings may be made without a significant reduction in patient adherence. This finding is in line with the assumption that the active factor responsible for improving adherence in guided versus unguided self-help interventions is that the participant is accompanied through the intervention. Providing instructions or detailed feedback on the content the participants worked on within the modules seems less critical for continued participant engagement.
However, the incremental value of offering feedback on demand compared to only adherence monitoring from an eCoach remains yet unclear. Within the adherence-focused guidance concept, it is hypothesized that feedback on demand is an important component so that the eCoach is seen as having the participant’s best interests at heart. Offering support may be an antecedent for creating an adherence-promoting relationship and, according to the supportive accountability model [
In the adherence-focused guidance study arm, monitoring adherence and sending a personalized reminder took up almost all the resources associated with this guidance format (approximately 1 hour per participant). In contrast, feedback on demand required much less resources. Hence, the question arises whether automatic reminders sent from the system on behalf of the eCoach have, in combination with feedback on demand, a similar effect on adherence, while requiring even less resources. Other studies have already shown positive effects on treatment adherence through automatic reminders [
If feedback on demand from a health professional is not a necessary component to achieve sufficient treatment adherence, adherence monitoring may also be performed by nonprofessionals. This would improve cost-effectiveness and dissemination. The eCoaches’ qualification level was not found to significantly influence treatment efficacy in Internet interventions for a range of conditions [
However, reducing human contact in Internet interventions can also entail potential risks for participants. Without content-related feedback, the eCoach may not become aware of problems participants may experience during the training. Thus, the risk for negative effects with Internet interventions for some individuals has the potential to be higher when receiving adherence-focused guidance instead of content-focused guidance. For this reason, negative effects should be investigated in future studies that compare different guidance formats [
Likewise, the guidance format could also influence the acceptance and attractiveness of Internet interventions, and thereby be important for dissemination. Therefore, varying guidance formats in Internet interventions should also be evaluated in terms of attractiveness and general acceptance [
The preceding discussion alludes to the many factors that can influence guidance in Internet interventions. Our third research question moves the focus from intervention characteristics to participant characteristics. It aimed to identify participant characteristics as predictors of nonadherence in an Internet-based mobile-supported stress management intervention. Although the guidance formats significantly predicted nonadherence in this study, the predictive value of the variables targeted in this analysis is small. The final model only explained 9.4% of the interindividual differences in nonadherence rates. Other variables, purported in previous literature as predictive of treatment nonadherence in Internet interventions, may prove to be incrementally important and should be investigated further. These include, for example, primary motivation, intention to adhere, and self-efficacy, as well as computer savviness, Internet affinity, and usability in addition to system- and program-related variables [
This study has several limitations. First, the participants were not randomized to the three study arms. Hence, the differences between the guidance formats may be confounded by other differences between the studies.
Second, participants in the adherence-focused guidance group showed a significantly higher percentage of female users at baseline compared to participants in the administrative guidance group. Furthermore, the content-focused guidance sample had a significantly higher education level compared to the administrative guidance and adherence-focused guidance samples. These unsystematic variations may have contributed to better adherence rates in the adherence-focused guidance and content-focused guidance groups compared to the administrative guidance group. But gender and education level were not significantly associated with treatment nonadherence. However, the total sample was found to be highly educated, which may limit the generalization to the population.
Third, the adherence rates identified in this study refer to an Internet-based mobile-supported stress management intervention administered in RCTs. Several studies indicate that adherence rates to Internet interventions in the context of RCTs are higher than those available from open-access websites [
Fourth, generalizability of the treatment adherence rates in an Internet-based mobile-supported stress management intervention are further limited by only including employees with elevated stress levels in the studies. Thus, the findings of this study may not be relevant for settings with participants that were not preselected based on their stress level. However, the stress level was not significantly associated with lower adherence rates beyond the guidance formats.
Fifth, only one adherence measure was included in the analysis. Different adherence measures need to be used which capture the quality of engagement with an intervention to a greater extent, such as time on website, number of completed homework assignments, and diary entries [
Sixth, as in most predictor studies, the analyses in this study were exploratory without any presumptions about the relationship between the predictors and adherence in order to generate hypotheses [
This study has important implications for research and practice. Guidance with focus on treatment adherence has the potential to be helpful in keeping participants involved in the training and, at the same time, keeping coaching costs low. Evaluating the cost-effectiveness as well as the comparative efficacy of the different guidance formats should be a next step to further complement the findings of this study [
randomized controlled trial
This study was funded by the European Union (EFRE: ZW6-80119999, CCI 2007DE161PR001) and the BARMER GEK. Furthermore, we would like to acknowledge Elena Heber, Angelina Scheel, and Laura Iffländer for the study administration.
Dirk Lehr, Matthias Berking and David Ebert are stakeholders of the “Institute for Online Health Trainings“, that aims to transfer scientific knowledge related to the present research into routine healthcare.