Published on 16.09.13 in Vol 15, No 9 (2013): September
An Internet-Based Intervention to Promote Mental Fitness for Mildly Depressed Adults: Randomized Controlled Trial
Background: Depression is a worldwide problem warranting global solutions to tackle it. Enhancing well-being has benefits in its own right and could be a good strategy for preventing depression. Providing well-being interventions via the Internet may have synergetic effects.
Objective: Psyfit (“mental fitness online”) is a fully automated self-help intervention to improve well-being based on positive psychology. This study examines the clinical effects of this intervention.
Methods: We conducted a 2-armed randomized controlled trial that compared the effects of access to Psyfit for 2 months (n=143) to a waiting-list control condition (n=141). Mild to moderately depressed adults in the general population seeking self-help were recruited. Primary outcome was well-being measured by Mental Health Continuum-Short Form (MHC-SF) and WHO Well-being Index (WHO-5); secondary outcomes were depressive symptoms, anxiety, vitality, and general health measured by Center for Epidemiological Studies Depression Scale (CES-D), Hospital Anxiety and Depression Scale Anxiety subscale (HADS-A), and Medical Outcomes Study-Short Form (MOS-SF) vitality and general health subscales, respectively. Online measurements were taken at baseline, 2 months, and 6 months after baseline.
Results: The dropout rate was 37.8% in the Psyfit group and 22.7% in the control group. At 2-month follow-up, Psyfit tended to be more effective in enhancing well-being (nonsignificantly for MHC-SF: Cohen’s d=0.27, P=.06; significantly for WHO-5: Cohen’s d=0.31, P=.01), compared to the waiting-list control group. For the secondary outcomes, small but significant effects were found for general health (Cohen’s d=0.14, P=.01), vitality (d=0.22, P=.02), anxiety symptoms (Cohen’s d=0.32, P=.001), and depressive symptoms (Cohen’s d=0.36, P=.02). At 6-month follow-up, there were no significant effects on well-being (MHC-SF: Cohen’s d=0.01, P=.90; WHO-5: Cohen’s d=0.26, P=.11), whereas depressive symptoms (Cohen’s d=0.35, P=.02) and anxiety symptoms (Cohen’s d=0.35, P=.001) were still significantly reduced compared to the control group. There was no clear dose–response relationship between adherence and effectiveness, although some significant differences appeared across most outcomes in favor of those completing at least 1 lesson in the intervention.
Conclusions: This study shows that an online well-being intervention can effectively enhance well-being (at least in the short-term and for 1 well-being measure) and can help to reduce anxiety and depression symptoms. Further research should focus on increasing adherence and motivation, reaching and serving lower-educated people, and widening the target group to include people with different levels of depressive symptoms.
Trial Registration: Netherlands Trial Register (NTR) number: NTR2126; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=2126 (archived by WebCite at http://www.webcitation.org/6IIiVrLcO).
J Med Internet Res 2013;15(9):e200
Relevance of Well-Being
Depression is a highly prevalent problem worldwide , which underscores the global urgency of tackling this mental illness [ ]. The average lifetime prevalence estimates of major depressive episodes is 15%, with 1 in 20 people suffering from major depressive episodes at any given time [ ]. In addition, many more people present subclinical depressive symptoms, putting them at greater risk of going on to develop a full-blown mental disorder [ ]. The enhancement of well-being may be just as important in dealing with poor mental health as treating the symptoms of depression [ ]. Ample studies show that the level of well-being is a predictor of psychopathology, independent of the influence of negative affect [ , ]. The presence of well-being is also beneficial in its own right. Well-being is associated with a healthier and longer life [ , ], prosocial behavior, and maintenance of high-quality relationships with family and friends [ ]. Thus, well-being may play a role both in supporting health and human functioning and in reducing symptoms of mental illness.
Three different types of well-being have been identified in various studies. Subjective well-being refers to positive affect and/or life satisfaction , psychological well-being refers to the level of positive functioning, containing constructs such as meaning in life, goal setting, and mastery [ ], and social well-being contains constructs such as the level of social integration and social contribution [ ].
In the emerging field of positive psychology, interventions aimed at flourishing and positive functioning are being developed and evaluated. These interventions focus on individual competencies and strengths, rather than on mental symptoms and disorders . Examples of interventions are “counting your blessings” [ ], “practicing kindness” [ ], “expressing gratitude” [ , ], and “using your personal strengths” [ ]. Positive psychology also embraces other like-minded traditions, such as mindfulness [ ] and life review [ ].
Evidence of the effectiveness of positive psychological interventions was reviewed in 2 meta-analyses showing that these interventions enhance well-being and reduce depressive symptoms [, ]. In addition, positive psychological interventions may reach people who are otherwise more difficult to connect with. People presenting subclinical depression or other problems, such as stress and anxiety, may find a well-being approach more appealing than a problem-oriented approach [ ]. One of the main obstacles worldwide in mental health care is to reach people in need of treatment or prevention with good quality interventions [ ]. By their inherent appeal, positive psychological interventions may help to narrow this mental health gap.
Combining Positive Psychological Interventions and the Internet
Providing positive psychological interventions via the Internet may offer synergetic opportunities. The effectiveness of online problem-based interventions (eg, in reducing depressive symptoms or anxiety  or problematic alcohol use [ ]) has been demonstrated meta-analytically. Internet interventions, especially self-help interventions, may be more affordable and accessible for many people as opposed to face-to-face interventions [ ]. Therefore, online positive psychological interventions may offer the most suitable and effective strategy for reaching large target groups. To date, a literature review of the effectiveness of online positive psychological interventions has shown mixed results [ ]. In this review, 5 randomized controlled trials were included [ - ], of which 3 demonstrated enhanced well-being [ , , ] and 3 showed significant symptom reduction [ - ]. The authors concluded that there is still an “effectiveness gap between offline and online formats” in positive psychological interventions [ ]. In more recent studies of online positive psychological interventions, mixed results were also found. In a comprehensive study by Gander et al [ ], 9 online positive psychological interventions were compared to a placebo exercise. Significant improvement in psychological well-being was found for 8 exercises, but not at all time points, and 5 of 9 exercises reduced depressive symptoms [ ]. In the study of Layous et al [ ] the online variant of the “best possible selves” exercise was just as effective as the face-to-face variant, and more effective than the placebo intervention. Schueller and Parks [ ] showed that it might not always be effective to provide more exercises and content because a 2-exercise and a 4-exercise condition were more effective than the 6-exercise condition.
One of the reasons for a gap between face-to-face and online interventions could be the low adherence rates [, ]. Adherence may be related to effectiveness such that the more people adhere to an intervention, the larger the effect sizes are [ , ]. This supposed relationship needs further clarification in online positive psychological interventions [ ].
The primary goal of the current study was to examine the effectiveness of Psyfit, an online positive psychological intervention, in comparison with a waiting-list control group. This study distinguishes itself from former experimental research, which was limited to single-component interventions [, , ] or longer multiple and fixed-sequential interventions [ , , ], in focusing on a multicomponent, flexible, online intervention. The intervention resembles a toolbox where people can pick and mix their personal training program. From self-determination theory, it can be reasoned that this idea would promote autonomy in the participant, leading to more intrinsic motivation to follow the program [ ]. Indeed, experimental studies show that most people are better off with a tailored positive psychological intervention shaped to their personal preferences and needs [ , ].
We hypothesized that the online positive psychological intervention group would demonstrate a significant increase in well-being and a reduction in depressive and anxiety symptoms at 2- and 6-month follow-ups as compared to the control group. The second goal of the study was to examine the role of adherence. We hypothesized that the more people adhered to the intervention, the larger the effects would be. A third goal was to examine whether particular subgroups benefit more or less than others from the intervention (less vs more depressive symptoms at baseline, higher- vs lower-educated people, men vs women, older vs younger people). Because of the broad nature of the intervention, it was hypothesized that each of the subgroups was served equally well.
We conducted a randomized controlled trial (NTR2126) of the online positive psychological intervention Psyfit as compared to a waiting-list control group. Online measurements were measured at baseline, 2 months, and 6 months after starting the intervention. Details of the study design and the intervention are reported more extensively in the study protocol that was published previously , but a summary is presented subsequently. The study was approved by the Dutch Medical Ethics Committee for Mental Health Care (registration number 9218).
Psyfit is an online self-help intervention, without support from a therapist. Psyfit aims to enhance well-being by stimulating personal growth and positive functioning. A parallel goal is to reduce depressive and anxiety symptoms . The intervention is based on positive psychological principles and addresses strengths and personal competencies rather than mental problems and deficiencies. It incorporates evidence-based exercises based on positive psychology and elements stemming from mindfulness, cognitive behavioral therapy, and problem-solving therapy [ ]. In Psyfits’ communication message, an analogy is made with physical fitness. Similar to the saying “an apple a day keeps the doctor away,” Psyfit encourages people to complete their daily mental fitness training.
There are 6 modules in Psyfit, each containing a 4-lesson program: (1) personal mission statement and setting your goals, (2) positive emotions, (3) positive relations, (4) mindfulness, (5) optimistic thinking, and (6) mastering your life. Each week, the lesson consisted of psycho-education and a practical exercise. At the end of the week, participants received an email notifying them that the next lesson could be started. Participants could start or finish modules as they wished, as long as they were in sequence.
Participants allocated to the intervention group received an email with a personal username and password. After logging in, 2-month access to Psyfit was activated. Participants were advised to complete at least 1 module during the intervention period. Each module is a separate module on its own and may, in theory, improve well-being.
Participants in the control group were told they were on a waiting list for 6 months before they received their login codes for Psyfit. A waiting-list control group is ethically acceptable when there is no immediate risk or treatment indication , thus fitting the preventive nature of this study. Participants in the control group had unrestricted access to professional help, if needed.
Participants were recruited from the adult well-being-seeking population in the Netherlands in March and April 2010. The recruitment message was formulated positively (not with a focus on symptoms and problems): “Would you like to increase your mental fitness? Improve your mental fitness and participate in our study of an online self-help program.” Banners and advertisements were placed in free newspapers and on Facebook. Interested people registered at the Psyfit website () [ ] and subsequently received an email with information on the study and a link to the online consent form and baseline questionnaire. Adult participants (21 years and older) were included when informed consent was obtained and if they presented mild to moderate depressive symptoms as represented by a score of 10-24 on the Center for Epidemiologic Studies Depression Scale (CES-D) [ ] and a languishing or moderate level of well-being as measured with the Mental Health Continuum-Short Form (MHC-SF) in which the levels languishing, moderate, and flourishing can be distinguished [ ]. Furthermore, they had to have access to a computer and Internet and have sufficient knowledge of the Dutch language. Participants with serious depressive symptoms (CES-D score ≥25) or active suicidal thoughts or plans (question from the Web Screening Questionnaire [ ]) were excluded. Those meeting any of these exclusion criteria were advised to seek professional help. Those scoring too high on well-being according to the inclusion criteria were excluded, but were invited to do Psyfit after the study. Participants in both the experimental and control group had unlimited access to professional help, such as primary care and psychological support.
Randomization took place after baseline measurement and was carried out by using a computer-generated randomization list in blocks of 2, stratified by gender, education (higher/lower), and depression symptom level (CES-D scores 10-15 and 16-24).
Outcomes and Instruments
The primary outcome measure was well-being. This was assessed with 2 questionnaires: the MHC-SF which measures positive mental health in terms of subjective, psychological, and social well-being , and the World Health Organization 5-item Well-Being Index (WHO-5) [ , ], a short measure of overall well-being. Secondary outcomes were depressive symptoms, as measured with the CES-D [ , ]; anxiety symptoms, as measured with the Hospital Anxiety and Depression Scale, Anxiety subscale (HADS-A) [ ]; and general health and vitality, as measured with 2 subscales from the Medical Outcomes Study 36-Item Short Form (MOS SF-36) [ ]. All measurement instruments were self-reported and administered via email with a link to the questionnaire on the Internet. The instruments have been shown to have satisfactory reliability and validity (also see the protocol article for elaboration) [ ]. Participant satisfaction was measured using the 8-item Client Satisfaction Questionnaire short form (CSQ-8) [ ].
During the trial, usage data from the Web application containing log files were gathered, which enabled monitoring of intervention adherence.
Analyses were conducted following the intention-to-treat (ITT) principle. Accordingly, all participants were analyzed in the group to which they were allocated. Missing data at 2-month and 6-month follow-ups were imputed using the estimation maximization (EM) method in SPSS version 19 (IBM Corp, Armonk, NY, USA). Reporting of the results follows the guidelines of the Consolidated Standards of Reporting Trials (CONSORT) eHealth checklist . We applied unpaired t tests and logistic regression analyses to examine possible baseline differences between dropouts and nondropouts. A chi-square test (χ2) was used to determine possible differential loss to follow-up between experimental group and control group. Dropout was defined as completing the baseline and 2-month follow-up questionnaire, but not the 6-month follow-up questionnaire, or completing the baseline questionnaire, but neither of the follow-up questionnaires.
The effectiveness of Psyfit was examined by regression analyses. We used the clinical outcomes on continuous measures (MHC-SF, WHO-5, CES-D, HADS-A, and subscales of the MOS SF-36) as dependent variables for the 2- and 6-month follow-ups separately. The intervention dummy and the baseline measurements of the corresponding outcome variables were used as independent variables.
The size of the effect was estimated by using Cohen’s d. Cohen’s d is calculated as the difference between 2 means divided by the pooled standard deviation. A Cohen’s d of 0.5 indicates that the mean of the intervention group is half a standard deviation larger than the mean of the control group. Cohen’s d from 0.56 to 1.2 is a large effect size, 0.33 to 0.55 is moderate, and 0 to 0.32 is considered small . We calculated effect sizes (baseline to 2- and 6-month follow-ups, respectively) of each condition separately, and after that calculated the difference between experimental group and control group (∆d). As a sensitivity analysis, a completers-only analysis was conducted as well.
To examine the role of adherence, a dose–response relationship was analyzed. Five separate groups were made up of participants adhering to 0, 1, 2, 3, or 4 lessons from any module. Differences between these groups and a possible linear relation (time×group interaction) were investigated by means of a repeated measures ANOVA analysis. The levels of adherence were used as independent variables and the clinical outcomes at 2- and 6-month follow-ups as repeated measures. An unpaired t test was used to examine the difference between hardly adhered (adhered to 0 or 1 lesson) and at least some adherence (adhered to 2-4 lessons).
Finally, a moderator analysis was conducted to determine whether certain subgroups (men vs women; education level higher vs lower; mild vs moderate depressive symptoms: CES-D score ≤16 or >16; younger/older age group: ≤45 years or >45 years) benefited more or less from the intervention. This was done by regressing the outcome variable on the independent group variables, the condition dummy, and the interaction between subgroup and condition dummy while controlling for the baseline measurement.
For all analyses, we used 2-sided tests with a significance level of P<.05. We only used P<.10 for the adherence analysis because of the reduced number of participants and lower power.
Flow of Participants
shows the flow of participants. A total of 845 persons were interested in participating in the study and registered with the website. After giving informed consent and filling out the online screening form and baseline questionnaire, 284 people were included in the study (33.6%). Participants were randomly assigned to the intervention (n=143) or the waiting-list control group (n=141).
The demographic characteristics and baseline scores of the participants are shown in. The mean age of participants was 43.2 years (SD 11.8) and most were female (226/284, 79.6%). Participants were mostly highly educated (208/284, 73.2%) and most had paid employment (214/284, 75.4%). The mean CES-D depression score at baseline was slightly above the cut-off score of 16 (mean 16.80, SD 4.13) indicating a clinically relevant level of depressive symptoms [ ], and the MHC-SF score was below the Dutch national mean of 3.98 (mean 3.47, SD 0.63) [ ].
|Characteristic||Psyfit (n=143)||Control group (n=141)||All (N=284)|
|Age (years), mean, (SD)||43.5 (11.7)||42.8 (11.9)||43.2 (11.8)|
|Age categories, n (%)|
|21-34 years||36 (25.2)||37 (26.2)||73 (25.7)|
|35-64 years||100 (69.9)||101 (71.6)||201 (70.8)|
|65-100 years||7 (4.9)||3 (2.1)||10 (3.5)|
|Female, n (%)||114 (79.7)||112 (79.4)||226 (79.6)|
|Major negative life event (yes), n (%)||80 (55.9)||73 (51.8)||153 (53.9)|
|Education, n (%)|
|Lower (up to high school and middle-level applied education)||39 (27.3)||37 (26.2)||76 (26.8)|
|Higher (academic/professional)||104 (72.7)||104 (73.8)||208 (73.2)|
|Daily activities, n (%)|
|Paid employment||106 (74.1)||108 (76.6)||214 (75.4)|
|Unemployed/unable to work||20 (14.0)||10 (7.1)||30 (10.6)|
|Other (student, at home, retired)||17 (11.9)||23 (16.3)||40 (14.1)|
|Living situation, n (%)|
|With partner, with or without children||92 (64.3)||84 (59.6)||176 (62.0)|
|No partner, with or without children||45 (31.5)||48 (34.0)||93 (32.7)|
|Other (dorm or with parents)||6 (4.2)||9 (6.4)||15 (5.3)|
|Test scores, mean (SD)|
|MHC-SF (well-being)||3.45 (0.62)||3.50 (0.63)||3.47 (0.63)|
|WHO-5 (well-being)||10.81 (4.31)||11.52 (4.38)||11.17 (4.35)|
|CES-D (depression)||16.91 (4.16)||16.67 (4.11)||16.80 (4.13)|
|HADS-A (anxiety)||6.55 (2.86)||6.53 (2.96)||6.54 (2.91)|
|MOS SF subscale general health||18.57 (3.51)||17.95 (3.61)||18.26 (3.57)|
|MOS SF subscale vitality||14.46 (3.16)||14.62 (2.83)||14.54 (3.00)|
The response rate was 75.4% (214/284) at 2-month follow-up and 69.7% (198/284) at 6-month follow-up. The response rate was significantly higher in the control group compared to the Psyfit group at 2-month follow-up (84.4% vs 66.4%; χ21 = 12.3, P<.001) and 6-month follow-up (77.3% vs 62.2%; χ21 = 7.6, P=.01).
Tested at P<.05, dropout analysis revealed that there were several significant differences between dropouts and completers. Those who dropped out were more likely to be living in a dorm or with their parents (χ22 = 9.9, P=.002) and be of younger age (χ21 = 4.2, P=.04). No significant differences emerged between dropouts and completers regarding baseline symptoms. Examination of the interaction showed that dropouts in the control group scored significantly lower than dropouts in the Psyfit group on the MOS SF-36 general health subscale, indicating poorer health for dropouts in the control group (F1,280 = 6.48, P=.01).
Effects of the Intervention
presents the means and standard deviations for the outcome measures at 2- and 6-month follow-ups plus effect sizes and the results of the regression analysis in the EM imputed ITT sample. In , the results of Psyfit on the MHC-SF, WHO-5, and CES-D are depicted. From baseline to the 2-month follow-up, well-being (the main outcome measure) as measured with the MHC-SF was higher although this did not meet statistical significance (beta = 0.09, P=.06). A significant effect on the other well-being measure, the WHO-5, was found (beta = 0.12, P=.01). Between-group effects fell within the small range (MHC-SF: d=0.27; WHO-5: d=0.31). With regard to the secondary outcome measures, the intervention group showed a significant decline at 2-month follow-up in both depressive symptoms (beta = –0.13, P=.02) and anxiety (beta = –0.15, P=.001), and a significant improvement in both self-reported health (beta = 0.09, P=.01) and vitality (beta = 0.12, P=.02) versus the control group. Effect sizes were small (MOS SF general health: d=0.14; MOS SF vitality: d=0.22; HADS-A: d=0.32) to medium (CES-D: d=0.36). At 6-month follow-up, results were sustained for anxiety (beta = –0.17, P=.001) and depressive symptoms (beta = –0.13, P=.02), presenting medium effect sizes (both d=0.35), but were no longer significant for well-being, health, and vitality.
The same trends in effects emerged in the completers-only analysis: positive effects for all outcomes at 2-month follow-up and sustained effects for anxiety and depressive symptoms at 6-month follow-up (). However, the effect of well-being at 2-month follow-up as measured with the MHC-SF was not significant in the completers-only analysis (beta = 0.08, P=.14, d=0.19) nor were there significant effects for depressive symptoms at 6-month follow-up (beta = –0.10, P=.14, d=0.26). Vitality was improved at 2-month follow-up, but this did not meet statistical significance (beta = 0.11, P=.06, d=0.19).
|Measures||Psyfit (n=143)||Control group (n=141)||Linear regression analysis|
|MOS SF health|
|MOS SF vitality|
|Measures||Psyfita||Control groupb||Linear regression analysis|
|Mean||SD||d||Mean||SD||d||Beta||t (df)||P value||∆d|
|MOS SF health|
|MOS SF vitality|
On average, 1.39 modules were started (SD 1.45). Most participants (74/143, 51.7%) started 1 module (mode and median). Participants most often enrolled in the positive emotions module (51/143, 35.7%), before personal mission statement and setting your goals (44/143, 30.8%), mindfulness (31/143, 21.7%), and optimistic thinking (29/143, 20.3%). Positive relations (22/143, 15.4%) and mastering your life (22/143, 15.4%) were the least popular modules.
In, different adherence grades and their effect sizes are shown for the experimental group (ITT sample). Adherence was defined as completing zero (31/143, 21.7%), 1 (46/143, 32.6%), 2 (24/143, 16.8%), 3 (29/143, 20.3%), or 4 lessons (13/143, 9.1%) from 1 or more modules. In plots, there is no clear dose–response relationship (the more adherence, the larger the effect size) recognizable. Tested at P<.10, repeated measures analysis ( ) showed significant effects for the MHC-SF at both 2- and 6-month follow-ups after completing 2 lessons (F8,274 = 1.72, P=.09, Cohen’s d pre-post 1.25 and 1.18). Significant effects were found for the MOS-SF general health subscale at 2-month follow-up after completing 4 lessons (F8,274 = 2.16, P=.03, Cohen’s d pre-post 0.71) and for the MOS-SF vitality subscale at 2-month follow-up after adhering to 2 lessons (F8,274 = 1.75, P=.09, Cohen’s d pre-post 0.74). When examining hardly any adherence (completing ≤1 lesson) versus at least some adherence (completing ≥2 lessons), 2 comparisons appeared to be significant (not presented in and ). The effect sizes of the HADS-A anxiety scale at 2-month follow-up (d=0.16 vs d=0.43, t141 = 2.24, P=.03) and of the CES-D depression scale at 6-month follow-up (d=0.47 vs d=0.87, t141 = 1.84, P=.07) were significantly higher for the at least some adherence group compared to those who completed 1 lesson or less.
Results regarding participant satisfaction are based on the completers sample (93/95 filled in the satisfaction questionnaire). Regarding participant satisfaction, 40 of 93 participants (43.0%) in the experimental group expressed satisfaction with Psyfit. The other 57.0% (53/93) were indifferent and/or dissatisfied. In the questionnaire, we were unable to separate the indifferent and the dissatisfied categories. Psyfit was found to be somewhat helpful in dealing with problems by 47.3% of the participants (44/93). Almost 70% (65/93) of the participants would recommend Psyfit to friends, and 66.7% (62/93) would do Psyfit (again), if needed. Adherence was positively related to satisfaction rate (χ23 = 15.0, P=.02).
A moderator analysis with well-being (MHC-SF and WHO-5) and depression (CES-D) as dependent variables was conducted to explore which subgroups benefited more or less from the intervention. The only significant result was found for education level on the WHO-5 well-being measure at 6-month follow-up (d=0.46 for higher-educated participants vs d = –0.20 for lower-educated participants; F1,280 = 6.23, P=.01). Other nonsignificant trends for educational level were found on WHO-5 well-being at 2-month follow-up (d=0.42 vs d=0.02; F1,280 = 3.60, P=.06) and on the CES-D at 6-month follow-up (d=0.58 vs d = –0.13, F1,280 = 3.66, P=.06), suggesting that higher-educated people profited more from the intervention than lower-educated people on these measures. For age, a nonsignificant trend was found on the CES-D at 6-month follow-up (d=0.61 for participants older than 45 years vs d=0.09 for younger participants, F1,280= 3.62, P = .06), suggesting that the older group benefited more from Psyfit. For the other 2 potential moderators, gender and depression status, no significant or relevant interaction effects were found.
|Measures||<1 lesson (n=31)||1 lesson (n=46)||2 lessons (n=24)||3 lessons (n=29)||≥4 lessons (n=13)|
|Mean (SD)||d||Mean (SD)||d||Mean (SD)||d||Mean (SD)||d||Mean (SD)||d|
|Baseline||3.46 (0.60)||3.46 (0.70)||3.17 (0.58)||3.52 (0.52)||3.69 (0.62)|
|2-month||3.84 (0.61)||0.63||3.90 (0.64)||0.66||4.05 (0.81)||1.25||3.92 (0.68)||0.66||3.99 (1.04)||0.36|
|6-month||3.74 (0.69)||0.43||3.87 (0.79)||0.55||3.98 (0.78)||1.18||3.94 (0.76)||0.65||3.91 (1.09)||0.25|
|Baseline||10.00 (3.59)||11.74 (4.22)||9.75 (5.33)||10.41 (4.45)||12.31 (3.22)|
|2-month||12.00 (3.21)||0.59||12.83 (4.01)||0.26||13.34 (6.24)||0.62||12.09 (4.01)||0.40||14.69 (5.60)||0.52|
|6-month||11.56 (3.91)||0.42||13.28 (4.76)||0.34||13.29 (5.48)||0.65||12.80 (4.93)||0.51||14.55 (4.86)||0.54|
|Baseline||17.71 (4.19)||16.28 (3.71)||17.58 (4.74)||17.41 (4.33)||14.92 (3.73)|
|2-month||15.21 (6.21)||0.47||13.73 (5.77)||0.53||13.99 (9.74)||0.47||13.61 (6.77)||0.67||9.31 (5.92)||1.13|
|6-month||14.06 (6.51)||0.67||14.42 (7.67)||0.31||12.16 (8.56)||0.78||12.03 (7.94)||0.84||9.85 (5.90)||1.03|
|Baseline||6.90 (2.53)||6.52 (2.96)||7.04 (3.21)||6.00 (3.11)||6.15 (2.03)|
|2-month||6.28 (2.47)||0.25||6.22 (2.88)||0.10||5.58 (3.62)||0.35||4.71 (2.58)||0.45||5.00 (2.80)||0.47|
|6-month||6.16 (3.19)||0.26||6.01 (3.37)||0.16||6.08 (4.00)||0.26||4.62 (2.94)||0.46||4.61 (3.40)||0.55|
|MOS SF health|
|Baseline||18.52 (2.74)||18.46 (3.82)||19.00 (3.95)||19.14 (3.55)||17.00 (3.06)|
|2-month||18.98 (2.74)||0.17||19.05 (3.14)||0.17||19.47 (4.05)||0.12||19.48 (2.80)||0.11||19.31 (3.43)||0.71|
|6-month||17.82 (2.81)||–0.25||19.07 (3.29)||0.17||18.88 (4.41)||–0.03||19.06 (3.30)||0.02||17.85 (3.34)||0.27|
|MOS SF vitality|
|Baseline||14.03 (2.77)||15.04 (3.17)||13.96 (3.97)||14.00 (2.87)||15.38 (2.84)|
|2-month||15.20 (2.16)||0.48||15.83 (3.29)||0.24||16.83 (3.81)||0.74||15.21 (2.82)||0.43||15.85 (3.24)||0.16|
|6-month||15.03 (2.77)||0.36||15.90 (3.64)||0.25||15.85 (4.23)||0.46||16.01 (3.67)||0.61||15.62 (3.91)||0.07|
This randomized controlled trial examined the efficacy of an online self-help course aimed at promoting mental fitness and subsequent well-being. The results at 2-month follow-up show that intervention group participants made a significant improvement in the level of overall well-being on one measure (WHO-5) than participants in the waiting-list control group and a nonsignificant improvement on the other well-being measure (MHC-SF). Furthermore, general health and vitality level were significantly enhanced, and depression and anxiety symptoms were reduced in comparison to the waiting-list control group. At 6-month follow-up, effects were maintained for depression and anxiety. All effects were in the small to medium range. Adherence analysis revealed no clear dose–response effect, although some larger effects appeared for people receiving at least a minimal part of the intervention. For well-being (only WHO-5) and depression, somewhat larger effects were found for higher-educated participants.
Comparison With Previous Work
The effects of taking part in Psyfit are comparable with effects of similar positive psychological interventions in self-help format with regard to well-being (0.14-0.33 in a meta-analysis at immediate follow-up), but appear to be larger for depression on average (0.15 in the same meta-analysis) . When compared with separate studies of other online positive self-help interventions, the effect sizes at immediate follow-up in the current study are quite similar, and higher in some cases. For example, in the Seligman et al study [ ], exercises that worked well had effect sizes of 0.23 for well-being and 0.14-0.28 for depression. In the study of Shapira and Mongrain [ ], effect sizes of 0.03-0.18 were found for well-being and 0.09-0.30 for depression. Mitchell et al [ ] found effect sizes ranging from 0.05 to 0.29 for well-being and an effect size of -0.17 for depression. At longer-term follow-up, effect sizes of the current study were comparable or higher for depression (up to 0.33 for Seligman [ ], 0.42 for Shapira [ ], and -0.04 for Mitchell [ ]), and lower for well-being (up to 0.28 for Seligman [ ], up to 0.30 for Shapira [ ], and up to 0.33 for Mitchell [ ]).
Adherence and Acceptability
More than three-quarters (78.3%) of the participants in the intervention group completed at least 1 lesson in Psyfit. The personal mission statement and setting your goals module and the positive emotions module were the most popular modules, each chosen by one-third of the total sample. This can be considered a satisfactory adherence rate in general because it is comparable with, or higher than, adherence rates in other online self-help interventions . However, from a practical point of view, it is questionable whether this adherence rate deserves our endorsement because less than 10% fully adhered to 1 module (consisting of 4 lessons). The group who completed at least 1 lesson showed slightly larger effect sizes than the whole group analysis, indicating that following the self-help course could predict effect size to some extent, but these results were not convincing. The explanation for this is not immediately clear. Dose-response effects (the more adherence, the larger the effects) have been established in several online trials of mental health promotion [ ]. On the other hand, not every trial demonstrated such an adherence effect [ ], which is comparable to our results. Unfortunately, we collected no information on the reasons for quitting (or not even starting) the intervention. One possible explanation is that nonparticipants/dropouts felt disappointed, or not capable of proceeding with the intervention. We did determine that adherence rate was positively related to satisfaction rate. Alternatively, it could be argued that people stopped because they felt better and were no longer in need of help, which could explain the modest effects of adherence, implying that the nonadhering participants were doing better as well.
Still, the question of how to increase the engagement and adherence for this type of intervention is important. It is likely that by increasing adherence, people could benefit more. The incorporation of persuasive and gaming elements into an intervention may hold considerable promise because such elements may raise client satisfaction and can encourage participants to stick to the intervention [, , ]. Less than half (approximately 40%) of participants were satisfied with Psyfit, although a larger percentage (approximately 65%) would recommend the program to a friend or would do it again if needed. Persuasive elements may help to arouse and prolong participant motivation, which may ultimately contribute to sustainable behavioral change [ ]. Although Psyfit already contains some engaging and motivating elements, such as email reminding and self-monitoring instruments, other persuasive functionalities could be added to make it a truly interactive and personalized device, such as tailored feedback and action planning [ ]. In addition, minimal guidance (eg, telephone/email contact or coaching supervision) is worth considering because this may increase adherence and effectiveness. There is some experimental evidence showing that tailoring positive psychological exercises to needs and preferences can indeed effectively enhance intervention engagement and adherence [ ], although preference for a certain well-being activity appears to be not enough to guarantee intervention effectiveness [ ]. For this reason, it seems warranted to ensure a good person-activity fit; a kind of diagnosis to determine which well-being enhancing exercise suits the participant best [ ]. Features of the intervention (dosage and variety) as well as personal aspects, such as motivation and efficacy beliefs, should be taken into account [ ]. Lastly, it has been found that lower-educated groups often lack the more sophisticated Internet skills that are needed to participate in an online intervention [ ]. Increasing those skills in these groups might contribute to improved adherence and better performance in online interventions such as Psyfit.
A relatively slight increase in the overall level of health in a sizeable part of the population may have a larger preventive effect than targeting only the much smaller group of people who are already ill [, ] . This principle may apply even more for mental health and well-being because stigma surrounding the formal use of mental health services may deter people in need from seeking help [ ]. Considering that many people can be reached by Psyfit in a nonstigmatized way and in a relatively short length of time, even the small effect sizes that were demonstrated in this trial may contribute to an important improvement in population mental health. On the other hand, this trial only included a specific target group of people presenting mild to moderate depressive symptoms, which affects generalizability to the larger population. In a naturalistic study, it has been found that people seeking self-help interventions to improve well-being either show rather severe depression scores or hardly any signs of depression [ ], whereas the present study targeted people with mild depressive symptoms. It would be insightful to examine if Psyfit is also effective in these other groups in a way that could be generalized to a larger population.
A number of limitations in this study have to be recognized. First, there was a rather high attrition rate and differential dropout between the intervention arms. Although not an uncommon phenomenon in online trials , the dropout may have affected the results in a way that is not easy to predict. Thus, the results of our trial should be considered with caution. Second, the intervention used in this study was designed like a toolbox from which people could pick and mix their own personalized program. This can be considered a strength because it enables participants to tailor their own program, which is a unique feature of the intervention. However, it may also be considered a weakness because no reliable statements can be given about the comparative effectiveness of modules and other effective elements of the intervention. Third, we did not measure motivational level, self-efficacy, or readiness to change as is constructed, for example, in the Stages of Change theory by Prochaska and DiClemente [ ]. Therefore, we could not examine whether the more motivated and better-equipped people adhered better to the intervention and accordingly benefited more. Fourth, the study used a waiting-list control group. This means there was no blinding of participants and possible placebo effects could not be established. Fifth, the CSQ-8 [ ] contains ambiguous reply categories. As a result, some of the satisfaction rates were difficult to interpret. For further research, we recommend that these categories be adapted to an unambiguous Likert scale, for example. Sixth, the effect sizes that were found were mostly in the small range. The study was powered to detect medium-sized effects [ ] and may, therefore, be underpowered to detect these small changes (eg, the nonsignificant finding for well-being/MHC-SF at 2-month follow-up).
Conclusion and Recommendations for Practice and Research
To the best of our knowledge, this is the first study of an online positive psychological intervention with a flexible multicomponent format to demonstrate both small, significant effects on well-being (for one of the measures) and on symptoms of mental disorder. Regarding the implications for public health, Psyfit could be used as an open and highly accessible mental health promotion tool, disseminated by relevant lifestyle and health-related websites and Internet forums, or referred to by health care professionals.
Regarding the future research agenda for online positive psychological interventions, emphasis should be placed on (1) increasing adherence and motivation by using persuasive design and/or providing minimal support, (2) determining what works best for whom and ensure a good person–activity fit, (3) serving lower-educated people better, and (4) considering the use of other control groups to overcome the limitations of a waiting-list control group. The target group could be expanded to present more variety in the depressive symptom spectrum. This will likely help to strengthen the generalizability of these results to a larger group of people.
One of the strengths of this study was the uncomplicated recruitment of participants (845 interested people within a 5-week timeframe), whereas in other randomized controlled trials of online health interventions recruitment can be a serious challenge . As such, Psyfit and other online positive psychological interventions can be regarded as positive technology, a recently proposed concept that refers to the use of technology for improving the quality of personal experience by fostering positive emotions, self-empowerment, and social connectedness [ ]. These technologies have the potential to evolve further and become available to many more people. As nonstigmatizing and nonmedicalized tools to address the promotion of mental health and subsequent prevention of depression, they contain the promise of a sustained positive effect on health care and society.
We are thankful to Jolanda Lourens, Jorne Grolleman, Monique Hulsbergen, Katherina Martin Abello, and Iris Rosier for their help with the development of the Psyfit intervention, and to Roisin de Jong for the English language edit of this paper. Also, we would like to express our gratitude to all participants without whom our study would not have been possible. The study is funded by the Dutch Ministry of Health, Welfare and Sport.
LB was the project leader, conducted the analyses, and wrote the manuscript. MH and LB together took care of the recruitment of participants and data collection. GW, JK, BB, HR, JW, and EB were advisors in the project. JW and HR obtained funding for the study. All authors contributed to the design of the study, provided comments, and approved the final manuscript.
Conflicts of Interest
Trimbos Institute is the developer of Psyfit and has a share in the social venture that implements the intervention. Neither the authors working at the Trimbos Institute nor the institute itself derive financial income from the interventions.
Multimedia Appendix 1
CONSORT-EHEALTH checklist V1.6.2 .PDF File (Adobe PDF File), 1000KB
- Bromet E, Andrade LH, Hwang I, Sampson NA, Alonso J, de Girolamo G, et al. Cross-national epidemiology of DSM-IV major depressive episode. BMC Med 2011;9:90 [FREE Full text] [CrossRef] [Medline]
- Department of Mental Health and Substance Dependence, World Health Organization. Prevention and Promotion in Mental Health. Geneva: World Health Organization; 2002. URL: http://www.who.int/mental_health/media/en/545.pdf [accessed 2013-08-19] [WebCite Cache]
- Cuijpers P, Smit F. [Subclinical depression: a clinically relevant condition?]. Tijdschr Psychiatr 2008;50(8):519-528 [FREE Full text] [Medline]
- Keyes CL. Promoting and protecting mental health as flourishing: a complementary strategy for improving national mental health. Am Psychol 2007;62(2):95-108. [CrossRef] [Medline]
- Wood AM, Joseph S. The absence of positive psychological (eudemonic) well-being as a risk factor for depression: a ten year cohort study. J Affect Disord 2010 May;122(3):213-217. [CrossRef] [Medline]
- Keyes CL, Dhingra SS, Simoes EJ. Change in level of positive mental health as a predictor of future risk of mental illness. Am J Public Health 2010 Dec;100(12):2366-2371. [CrossRef] [Medline]
- Howell RT, Kern ML, Lyubomirsky S. Health benefits: meta-analytically determining the impact of well-being on objective health outcomes. Health Psychological Review 2007;1(1):83-136. [CrossRef]
- Pressman SD, Cohen S. Does positive affect influence health? Psychol Bull 2005 Nov;131(6):925-971. [CrossRef] [Medline]
- Veenhoven R. Happiness: also known as "life-satisfaction" and "subjective well-being". In: Land KC, Michalos AC, Sirgy MJ, editors. Handbook of Social Indicators and Quality of Life Research. Dordrecht: Springer Publishers; 2012.
- Ryff CD. Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology 1989;57(6):1069-1081. [CrossRef]
- Keyes CLM. Social well-being. Social Psychology Quarterly 1998 Jun;61(2):121. [CrossRef]
- Seligman ME, Csikszentmihalyi M. Positive psychology. An introduction. Am Psychol 2000 Jan;55(1):5-14. [Medline]
- Seligman MEP, Steen TA, Park N, Peterson C. Positive psychology progress: empirical validation of interventions. Am Psychol 2005;60(5):410-421. [CrossRef] [Medline]
- Otake K, Shimai S, Tanaka-Matsumi J, Otsui K, Fredrickson BL. Happy people become happier through kindness: A counting kindness intervention. Journal of Happiness Studies 2006;7(3):361-375. [CrossRef]
- Sheldon KM, Lyubomirsky S. How to increase and sustain positive emotion: The effects of expressing gratitude and visualizing best possible selves. The Journal of Positive Psychology 2006 Apr;1(2):73-82. [CrossRef]
- Brown KW, Ryan RM. The benefits of being present: mindfulness and its role in psychological well-being. J Pers Soc Psychol 2003 Apr;84(4):822-848. [Medline]
- Bohlmeijer E, Roemer M, Cuijpers P, Smit F. The effects of reminiscence on psychological well-being in older adults: a meta-analysis. Aging Ment Health 2007 May;11(3):291-300. [CrossRef] [Medline]
- Sin NL, Lyubomirsky S. Enhancing well-being and alleviating depressive symptoms with positive psychology interventions: a practice-friendly meta-analysis. J Clin Psychol 2009 May;65(5):467-487. [CrossRef] [Medline]
- Bolier L, Haverman M, Westerhof GJ, Riper H, Smit F, Bohlmeijer E. Positive psychology interventions: a meta-analysis of randomized controlled studies. BMC Public Health 2013;13:119 [FREE Full text] [CrossRef] [Medline]
- Parks AC, Schueller SM, Tasimi A. Increasing happiness in the general population: empirically supported self-help? In: Boniwell I, David S, editors. Oxford Handbook of Happiness. Oxford: Oxford University Press; 2012.
- Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ 2004 Nov;82(11):858-866 [FREE Full text] [CrossRef] [Medline]
- Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
- Riper H, van Straten A, Keuken M, Smit F, Schippers G, Cuijpers P. Curbing problem drinking with personalized-feedback interventions: a meta-analysis. Am J Prev Med 2009 Mar;36(3):247-255. [CrossRef] [Medline]
- Muñoz RF. Using evidence-based internet interventions to reduce health disparities worldwide. J Med Internet Res 2010;12(5):e60 [FREE Full text] [CrossRef] [Medline]
- Mitchell J, Vella-Brodrick D, Klein B. Positive psychology and the Internet: A mental health opportunity. EJAP 2010 Dec 30;6(2):30-41. [CrossRef]
- Abbott JA, Klein B, Hamilton C, Rosenthal A. The impact of online resilience training for sales managers on wellbeing and performance. E-Journal of Applied Psychology 2009;5(1):89-95. [CrossRef]
- Mitchell J, Stanimirovic R, Klein B, Vella-Brodrick D. A randomised controlled trial of a self-guided internet intervention promoting well-being. Computers in Human Behavior 2009 May;25(3):749-760. [CrossRef]
- Parks-Sheiner AC. Positive Psychotherapy: Building a Model of Empirically Supported Self-Help. Cambridge: ProQuest Information & Learning; 2009.
- Seligman ME, Steen TA, Park N, Peterson C. Positive psychology progress: empirical validation of interventions. Am Psychol 2005;60(5):410-421. [CrossRef] [Medline]
- Shapira LB, Mongrain M. The benefits of self-compassion and optimism exercises for individuals vulnerable to depression. The Journal of Positive Psychology 2010 Sep;5(5):377-389. [CrossRef]
- Gander F, Proyer RT, Ruch W, Wyss T. Strength-based positive interventions: further evidence for their potential in enhancing well-being and alleviating depression. J Happiness Stud 2012 Sep 4;14(4):1241-1259. [CrossRef]
- Layous K, Katherine Nelson S, Lyubomirsky S. What is the optimal way to deliver a positive activity intervention? The case of writing about one’s best possible selves. J Happiness Stud 2012 May 6;14(2):635-654. [CrossRef]
- Schueller SM, Parks AC. Disseminating self-help: positive psychology exercises in an online trial. J Med Internet Res 2012;14(3):e63 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
- Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
- Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to Web-based interventions. J Med Internet Res 2012;14(6):e152 [FREE Full text] [CrossRef] [Medline]
- Ryan RM, Deci EL. Self-regulation and the problem of human autonomy: does psychology need choice, self-determination, and will? J Pers 2006 Dec;74(6):1557-1585. [CrossRef] [Medline]
- Lyubomirsky S, Dickerhoof R, Boehm JK, Sheldon KM. Becoming happier takes both a will and a proper way: an experimental longitudinal intervention to boost well-being. Emotion 2011 Apr;11(2):391-402. [CrossRef] [Medline]
- Schueller SM. To each his own well-being boosting intervention: using preference to guide selection. The Journal of Positive Psychology 2011 Jul;6(4):300-313. [CrossRef]
- Bolier L, Haverman M, Kramer J, Boon B, Smit F, Riper H, et al. Internet-based intervention to promote mental fitness in mildly depressed adults: design of a randomized controlled trial. JMIR Res Protoc 2012;1(1):e2. [CrossRef] [Medline]
- Parks AC, Della Porta MD, Pierce RS, Zilca R, Lyubomirsky S. Pursuing happiness in everyday life: the characteristics and behaviors of online happiness seekers. Emotion 2012 Dec;12(6):1222-1234. [CrossRef] [Medline]
- Walburg JA. Mentaal Vermogen - Investeren in geluk [Mental Capital - The Investment in Happiness]. Amsterdam: Nieuw Amsterdam; 2008.
- Mohr DC, Spring B, Freedland KE, Beckner V, Arean P, Hollon SD, et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother Psychosom 2009;78(5):275-284. [CrossRef] [Medline]
- psyfit.nl. URL: http://www.psyfit.nl/ [accessed 2012-01-27] [WebCite Cache]
- Haringsma R, Engels GI, Beekman AT, Spinhoven P. The criterion validity of the Center for Epidemiological Studies Depression Scale (CES-D) in a sample of self-referred elders with depressive symptomatology. Int J Geriatr Psychiatry 2004 Jun;19(6):558-563. [CrossRef] [Medline]
- Lamers SM, Westerhof GJ, Bohlmeijer ET, ten Klooster PM, Keyes CL. Evaluating the psychometric properties of the Mental Health Continuum-Short Form (MHC-SF). J Clin Psychol 2011 Jan;67(1):99-110. [CrossRef] [Medline]
- Donker T, van Straten A, Marks I, Cuijpers P. A brief Web-based screening questionnaire for common mental disorders: development and validation. J Med Internet Res 2009;11(3):e19 [FREE Full text] [CrossRef] [Medline]
- Bech P, Olsen LR, Kjoller M, Rasmussen NK. Measuring well-being rather than the absence of distress symptoms: a comparison of the SF-36 Mental Health subscale and the WHO-Five Well-Being Scale. Int J Methods Psychiatr Res 2003;12(2):85-91. [Medline]
- Henkel V, Mergl R, Kohnen R, Maier W, Möller HJ, Hegerl U. Identifying depression in primary care: a comparison of different methods in a prospective cohort study. BMJ 2003 Jan 25;326(7382):200-201 [FREE Full text] [Medline]
- Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1977 Jun 01;1(3):385-401. [CrossRef]
- Spinhoven P, Ormel J, Sloekers PP, Kempen GI, Speckens AE, Van Hemert AM. A validation study of the Hospital Anxiety and Depression Scale (HADS) in different groups of Dutch subjects. Psychol Med 1997 Mar;27(2):363-370. [Medline]
- Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992 Jun;30(6):473-483. [Medline]
- de Brey H. A cross-national validation of the client satisfaction questionnaire: the Dutch experience. Eval Program Plann 1983;6(3-4):395-400. [Medline]
- Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
- Lipsey MW, Wilson DB. The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. Am Psychol 1993 Dec;48(12):1181-1209. [Medline]
- van der Zanden R, Kramer J, Gerrits R, Cuijpers P. Effectiveness of an online group course for depression in adolescents and young adults: a randomized trial. J Med Internet Res 2012;14(3):e86 [FREE Full text] [CrossRef] [Medline]
- Fogg BJ. Persuasive Technology: Using Computers to Change What We Think and Do. San Francisco: Morgan Kaufmann; 2003.
- Lehto T, Oinas-Kukkonen H. Persuasive features in Web-based alcohol and smoking interventions: a systematic review of the literature. J Med Internet Res 2011;13(3):e46 [FREE Full text] [CrossRef] [Medline]
- Lyubomirsky S, Layous K. How do simple positive activities increase well-being? Current Directions in Psychological Science 2013 Jan 22;22(1):57-62. [CrossRef]
- van Deursen AJ, van Dijk JA. Internet skills performance tests: are people ready for eHealth? J Med Internet Res 2011;13(2):e35 [FREE Full text] [CrossRef] [Medline]
- Huppert FA. A new approach to reducing disorder and improving well-being. Perspectives on Psychological Science 2009;4(1):108-111. [CrossRef]
- Rose G. Rose's Strategy of Preventive Medicine: The Complete Original Text. Oxford: Oxford University Press; 2008.
- Schomerus G, Angermeyer MC. Stigma and its impact on help-seeking for mental disorders: what do we know? Epidemiol Psichiatr Soc 2008;17(1):31-37. [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Prochaska JO, DiClemente CC. Stages of change in the modification of problem behaviors. Prog Behav Modif 1992;28:183-218. [Medline]
- Murray E, Khadjesari Z, White IR, Kalaitzaki E, Godfrey C, McCambridge J, et al. Methodological challenges in online trials. J Med Internet Res 2009;11(2):e9 [FREE Full text] [CrossRef] [Medline]
- Riva G, Baños RM, Botella C, Wiederhold BK, Gaggioli A. Positive technology: using interactive technologies to promote positive functioning. Cyberpsychol Behav Soc Netw 2012 Feb;15(2):69-77. [CrossRef] [Medline]
|CES-D: Center for Epidemiologic Studies Depression Scale|
|CSQ-8: 8-item Client Satisfaction Questionnaire-short form|
|EM: estimation maximization|
|HADS-A: Hospital Anxiety and Depression Scale, Anxiety subscale|
|MHC-SF: Mental Health Continuum-Short Form|
|MOS SF-36: Medical Outcomes Study 36-Item Short Form|
|WHO-5: World Health Organization 5-item Well-Being Index|
Edited by G Eysenbach; submitted 06.03.13; peer-reviewed by A Parks, A Bauer, YS Bin, K Kaipainen, A Jarden; comments to author 07.04.13; revised version received 20.06.13; accepted 30.07.13; published 16.09.13
©Linda Bolier, Merel Haverman, Jeannet Kramer, Gerben J Westerhof, Heleen Riper, Jan A Walburg, Brigitte Boon, Ernst Bohlmeijer. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.09.2013.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.