This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
In previous research, variables such as age, education, treatment credibility, and therapeutic alliance have shown to affect patients’ treatment adherence and outcome in Internet-based psychotherapy. A more detailed understanding of how such variables are associated with different measures of adherence and clinical outcomes may help in designing more effective online therapy.
The aims of this study were to investigate demographical, psychological, and treatment-specific variables that could predict dropout, treatment adherence, and treatment outcomes in a study of online relaxation for mild to moderate stress symptoms.
Participant dropout and attrition as well as data from self-report instruments completed before, during, and after the online relaxation program were analyzed. Multiple linear and logistical regression analyses were conducted to predict early dropout, overall attrition, online treatment progress, number of registered relaxation exercises, posttreatment symptom levels, and reliable improvement.
Dropout was significantly predicted by treatment credibility, whereas overall attrition was associated with reporting a focus on immediate consequences and experiencing a low level of intrinsic motivation for the treatment. Treatment progress was predicted by education level and treatment credibility, whereas number of registered relaxation exercises was associated with experiencing intrinsic motivation for the treatment. Posttreatment stress symptoms were positively predicted by feeling external pressure to participate in the treatment and negatively predicted by treatment credibility. Reporting reliable symptom improvement after treatment was predicted by treatment credibility and therapeutic bond.
This study confirmed that treatment credibility and a good working alliance are factors associated with successful Internet-based psychotherapy. Further, the study showed that measuring adherence in different ways provides somewhat different results, which underscore the importance of carefully defining treatment adherence in psychotherapy research. Lastly, the results suggest that finding the treatment interesting and engaging may help patients carry through with the intervention and complete prescribed assignments, a result that may help guide the design of future interventions.
Clinicaltrials.gov NCT02535598; http://clinicaltrials.gov/ct2/show/NCT02535598 (Archived by WebCite at http://www.webcitation.org/6fl38ms7y).
It is well established that therapist-guided Internet-based cognitive behavior therapy (ICBT) and other behavioral interventions can be effective in improving psychological symptoms and well-being [
Adherence to Internet-based treatments can be operationalized in different ways, including working with the intervention by reading texts and watching video clips on a webpage or adhering to the behavioral prescriptions in everyday life (eg, by completing homework assignments) [
Self-determination theory has been applied to health behaviors and it seems that different forms of motivation may have different effects on treatment adherence [
Previous studies have shown that adherence to an intervention, as well as treatment outcome, may be influenced by demographical variables, such as gender, age, and education [
Finally, patients’ belief in a treatment may have a very strong effect on the outcome as seen in studies on placebo effects [
This study aimed at investigating variables that may predict three different types of outcome variables in ICBT: (1) dropout and attrition from treatment, (2) treatment adherence, and (3) clinical outcomes. More specifically, the aim was to assess the predictive value of different background variables as well as the variables time perspective, treatment credibility, motivation, and therapeutic bond on early dropout, attrition, treatment progress, adherence to behavioral prescriptions, posttreatment symptoms, and reliable improvement in a previously conducted study of a brief online stress management treatment. A secondary aim was to investigate whether treatment adherence could predict clinical outcomes.
Data for this study were retrieved from a previously conducted study on Internet-based relaxation training for people with mild to moderate stress and anxiety symptoms [
Early dropout (yes/no) was assessed by counting the number of participants who discontinued the study before completing the first week of the treatment program; attrition (yes/no) was assessed by counting the total number of participants who discontinued the study before the posttreatment assessment. Treatment adherence was divided into the two variables treatment progress and registered exercises, both on continuous scales. Treatment progress was assessed by measuring how much of the Web-based treatment material (eg, texts, examples, assignments) each participant accessed before dropping out or completing the treatment. Because the treatment consisted of 25 such items, this variable ranged from zero (not accessed the treatment) to 25 (accessed the whole treatment). Registered exercises were measured by the mean number of prescribed exercises of applied relaxation that the participant had registered on the webpage each week. The weekly number of registered exercises ranged from zero (not completed any exercises) to 14 (completed all prescribed exercises). As a measure of stress symptoms and treatment outcome, the Perceived Stress Scale (PSS) [
The predictor variables consisted of the background variables age, gender, level of education (primary/secondary/university), occupation (student/unemployed/employed/sick leave/retired), and computer expertise (low/intermediate/high). In order to facilitate interpretation, education and occupation were transformed into dichotomous variables (nonuniversity vs university; employed/student/retired vs unemployed/sick leave). Four psychological predictor variables were collected by self-report instruments at baseline: time perspective, treatment credibility, and internal and external motivation. Intrinsic motivation and therapeutic bond were measured at midtreatment and stress symptoms were measured both at baseline as a predictor variable and at postmeasurement as an outcome variable. Because a previous analysis [
Time perspective was measured with the Zimbardo Time Perspective Inventory Short Form (ZTPI) [
Treatment credibility was measured with the Treatment Credibility Scale (TCS), which is often used in studies of Internet interventions and is an adaptation from Borkovec and Nau [
Internal (ie, identified and integrated motivation) and external motivation were measured with the Treatment Self-Regulation Questionnaire (TSRQ) [
Intrinsic motivation was measured with the Intrinsic Motivation Inventory (IMI) [
Therapist bond was measured with the Working Alliance Questionnaire Short Form (WAI) [
Before analysis, data were screened for outliers and normality, linearity, and homoscedasticity were evaluated by scrutinizing the residual scatterplots between predicted variables and errors of prediction and found adequate. Because subscales were entered into the analyses, multicollinearity was assessed by analyzing the variance inflation factor for each predictor variable and found to be nonproblematic.
Reliable improvement was computed by dividing the difference between the pretreatment and posttreatment scores by the standard error of the difference between the two scores. If the Reliable Change Index was greater than 1.96, a change of that magnitude would not be expected due to the unreliability of the measure [
Modeled after deGraaf et al [
Of the 157 participants in this study, 115 (73.2%) were women and the mean age was 34.5 (SD 13.1) years. The background variables education, occupation, and computer expertise are shown in
Background predictor variables (N=157).
Factor | n (%) | |
|
|
|
|
Primary | 8 (5.1) |
|
Secondary | 57 (36.3) |
|
University | 92 (58.6) |
|
|
|
|
Studying | 43 (27.4) |
|
Unemployed | 9 (5.7) |
|
Employed | 81 (51.6) |
|
Sick leave | 16 (10.2) |
|
Retired | 8 (5.1) |
|
|
|
|
Low | 54 (34.4) |
|
Intermediate | 51 (32.5) |
|
High | 52 (33.1) |
There were no significant differences for any of the predictor variables between the treatment groups of the original study. Of the background variables, only education and occupation were significant predictors for any outcome variable in the initial bivariate regression analyses. Of the self-reported psychological variables, the TSRQ-IM showed markedly higher standard deviation compared to other variables and it was the only variable that failed to significantly predict any outcome variable, so it was removed from further analyses. The results of these bivariate analyses for each outcome variable can be found in
The multivariate logistic regression analyses showed that early dropout could be significantly negatively predicted by the TCS (B=–0.14, χ2
1=10.5,
Significant predictor variables for early dropout and attrition after backward deletion (N=157).
Treatment dropouta | Cox-Snell |
Nagelkerke |
B (SE) | χ2 1 |
|
OR (95% CI) | |
|
.18 | .39 |
|
|
|
|
|
|
TCS |
|
|
–0.14 (0.04) | 10.5 | .001 | 0.87 (0.80-0.95) |
|
.19 | .28 |
|
|
|
|
|
|
Baseline stress symptoms |
|
|
0.08 (0.05) | 3.2 | .05 | 1.08 (1.00-1.18) |
|
ZTPI Hedonistic |
|
|
0.32 (0.10) | 10.3 | .001 | 1.37 (1.13-1.66) |
|
IMI |
|
|
–0.06 (0.03) | 5.7 | .02 | 0.94 (0.90-0.99) |
aIMI: Intrinsic Motivation Inventory; TCS: Treatment Credibility Scale; ZTPI: Zimbardo Time Perspective Inventory.
After controlling for level of support, treatment progress was positively predicted by level of education (beta=.24,
Significant predictor variables for treatment adherence after backward deletion (N=157).
Treatment adherencea |
|
B (SE) | β |
|
|
|
|
.36 |
|
|
|
|
|
|
Enhanced support |
|
2.84 (0.88) | .33 | 3.23 (153) | .002 |
|
University education |
|
1.95 (0.84) | .24 | 2.33 (153) | .02 |
|
TCS |
|
0.15 (0.04) | .35 | 3.36 (153) | .001 |
|
.13 |
|
|
|
|
|
|
IMI |
|
0.10 (0.04) | .29 | 2.43 (155) | .02 |
aIMI: Intrinsic Motivation Inventory; TCS: Treatment Credibility Scale.
Posttreatment stress symptoms were significantly and positively predicted by the baseline stress symptoms (beta=.47,
Significant predictor variables for post treatment stress symptoms after stepwise deletion (n=96).
Predictor variablea |
|
B (SE) | β |
|
|
Postmeasurement stress symptoms | .42 |
|
|
|
|
Baseline stress symptoms |
|
0.53 (0.12) | .47 | 4.43 | <.001 |
TSRQ-EM |
|
0.48 (0.20) | .25 | 2.40 | .02 |
TCS |
|
–0.28 (0.08) | –.35 | 3.53 | .001 |
aTCS: Treatment Credibility Scale; TSRQ-EM: Treatment Self-Regulation Questionnaire Extrinsic Motivation.
Significant predictor variables for reliable improvement after stepwise deletion (n=96).
Predictor variablea | Cox-Snell |
Nagelkerke |
B (SE) | χ2 1 |
|
OR (95% CI) |
Reliably improved | .32 | .47 |
|
|
|
|
Baseline stress symptoms |
|
|
0.11 (0.05) | 4.5 | .03 | 1.12 (1.01-1.25) |
TCS |
|
|
0.09 (0.05) | 3.3 | .05 | 1.10 (1.00-1.20) |
WAI total |
|
|
0.14 (0.07) | 3.9 | .049 | 1.15 (1.01-1.32) |
aTCS: Treatment Credibility Scale; WAI: Working Alliance Inventory.
In bivariate regression analyses and after controlling for baseline PSS score, posttreatment stress symptoms was significantly negatively predicted by both treatment progress (beta=–.31,
The analyses showed that different ways of operationalizing (ie, measuring) dropout, treatment adherence, and treatment outcomes was related to somewhat different predictor variables, something that may be important to consider in psychotherapy process research. First, there was a difference in that early treatment dropout could be predicted by treatment credibility, whereas further attrition was predicted by a personality pattern of focusing of immediate consequences and by finding the treatment unengaging, two variables that are probably connected. People whose behavior is generally governed by immediate consequences and reward may find an online behavioral treatment largely consisting of abstract instructions and texts unsatisfying and boring. This pattern suggests that some people with low expectations of the treatment start the program, but very soon realize it does not suit them and quit. Whether these participants should receive more motivational support from therapists to stay in the online treatment or efforts should be made to guide them to other forms of treatment needs further investigation. Participants who dropped out later in the treatment reported a lower tendency to focus on future goals and also experienced the treatment as unrewarding. These results are in line with previous results showing that being able to focus on the future and to postpone rewards is associated with higher levels of education and health behaviors [
Treatment adherence showed a somewhat similar pattern to dropout with online treatment progress predicted by education and treatment credibility. Thus, working with the online material was associated with a probable familiarity of working with texts and thinking in abstract terms. Treatment credibility may correspond to a familiarity and interest in using the Internet for learning about health behaviors and perhaps previous positive experiences of online courses. Treatment credibility may also represent participants’ belief in the treatment model and specifically in using relaxation to target stress symptoms. Finally, the association between treatment credibility and treatment progress may represent the different reasons why people undergo Internet-based psychotherapy; for some it is a preferred choice, whereas for others it may be the only available option. Completion of homework assignments, similar to staying in the treatment program, was predicted by finding the treatment engaging. That the IMI was significantly associated with adherence in this study may be partly explained by the nature of relaxation exercises that may actually be pleasant in contrast to exposure exercises, for example. It is worth noting that the instrument used to measure intrinsic motivation, the IMI, is designed to measure different forms of intrinsic reward and comprise items concerning experiences of interest, enjoyment, and meaningfulness [
Among participants who did not dropout and remained in the study, treatment outcome, as measured by the posttreatment stress symptoms, was significantly positively predicted by baseline stress symptoms, but also by the TSRQ-EM measuring external motivation. External motivation corresponds to feeling pressured by others, mostly in a negative way, to complete tasks. In previous studies, external motivation has been associated with difficulties in sustaining behavior and a reliance on accountability [
In this study, many of the proposed predictor variables could not significantly predict the outcome variables in the multivariate analyses. This suggests that several of the predictor variables covaried to a large degree; therefore, finding the best predictor variables is difficult. The multivariate analyses with backward deletion resulted in models with acceptable levels of model fit except for registered exercises (
This study has a number of limitations. First, several of the predictor and outcome variables were difficult to measure accurately. For example, the measurement of registered exercises was constructed to assess the treatment’s effective mechanism, but may have failed to fully capture changes in participants’ everyday behavior. The intervention encouraged participants to conduct relaxation exercises and highly engaged participants may have done so without registering on the webpage. Further, several of the self-report instruments have not been used in psychotherapy research before and their psychometric properties in this context are unknown. For example, the internal reliability of some questionnaires seemed to be somewhat lower in this study compared to previously reported figures. Second, a more complex design with repeated measurements of adherence could have been conducted to show causal mediation, but this would also have demanded a much larger sample size. Third, the lack of a control group or a face-to-face treatment condition limits the generalizability of the results. Therefore, whether the conclusions from this study are valid for other treatment modalities are unknown. Finally, and most importantly, there was large dropout (39%) at postmeasurement and although investigating dropout from Internet-based interventions was one of the aims of this study, it also meant loss of follow-up data. This also means that the conclusions regarding predictors of clinical outcome are only valid for participants who stay in the study or treatment, which limits the generalizability of the results. A more thorough analysis of participants who dropped out may have resulted in a better understanding of this group.
In conclusion, this study confirms the importance of treatment credibility and working alliance in Internet-based psychotherapy and also suggests that experiencing intrinsic reward from participating in the treatment may be important. In contrast, external pressure to try online therapy may be counterproductive and lead to worse outcomes. Apart from identifying people who believe in the online treatment format and continue to explore the best methods for online therapist support, it may also be valuable to further investigate what makes participants engage in an intervention and what features makes the intervention interesting and meaningful [
Identified candidate predictor variables from the bivariate analyses for nominal outcome variables.
Identified candidate predictor variables from the bivariate analyses for continous outcome variables.
Internet-based cognitive behavior therapy
internal motivation
Intrinsic Motivation Inventory
Perceived Stress Scale
self-determination theory
Treatment Credibility Scale
Treatment Self-Regulation Questionnaire
This study was funded by the Swedish government U-CARE grant to Uppsala University.
None declared.