Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 13.05.19 in Vol 21, No 5 (2019): May

Preprints (earlier versions) of this paper are available at, first published Jun 22, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Reducing Stress and Preventing Depression (RESPOND): Randomized Controlled Trial of Web-Based Rumination-Focused Cognitive Behavioral Therapy for High-Ruminating University Students

    1SMART Lab, Mood Disorders Centre, School of Psychology, University of Exeter, Exeter, United Kingdom

    2College of Life and Environmental Sciences (CLES), School of Psychology, University of Exeter, Exeter, United Kingdom

    *all authors contributed equally

    Corresponding Author:

    Edward Watkins, PhD

    SMART Lab, Mood Disorders Centre

    School of Psychology

    University of Exeter

    Sir Henry Wellcome Building for Mood Disorders Research

    Queens Drive

    Exeter, EX4 4QG

    United Kingdom

    Phone: 44 (0) 139272 ext 4692



    Background: Prevention of depression is a priority to reduce its global disease burden. Targeting specific risk factors, such as rumination, may improve prevention. Rumination-focused Cognitive Behavioral Therapy (RFCBT) was developed to specifically target depressive rumination.

    Objective: The primary objective of this study was to test whether guided Web-based RFCBT (i-RFCBT) would prevent the incidence of major depression relative to usual care in UK university students. The secondary objective was to test the feasibility and estimated effect sizes of unguided i-RFCBT.

    Methods: To address the primary objective, a phase III randomized controlled trial was designed and powered to compare high risk university students (N=235), selected with elevated worry/rumination, recruited via an open access website in response to circulars within universities and internet advertisements, randomized to receive either guided i-RFCBT (interactive Web-based RFCBT, supported by asynchronous written Web-based support from qualified therapists) or usual care control. To address the secondary objective, participants were also randomized to an adjunct arm of unguided (self-administered) i-RFCBT. The primary outcome was the onset of a major depressive episode over 15 months, assessed with structured diagnostic interviews at 3 (postintervention), 6, and 15 months post randomization, conducted by telephone, blind to the condition. Secondary outcomes of symptoms of depression and anxiety and levels of worry and rumination were self-assessed through questionnaires at baseline and the same follow-up intervals.

    Results: Participants were randomized to guided i-RFCBT (n=82), unguided i-RFCBT (n=76), or usual care (n=77). Guided i-RFCBT reduced the risk of depression by 34% relative to usual care (hazard ratio [HR] 0.66, 95% CI 0.35 to 1.25; P=.20). Participants with higher levels of baseline stress benefited most from the intervention (HR 0.43, 95% CI 0.21 to 0.87; P=.02). Significant improvements in rumination, worry, and depressive symptoms were found in the short-to-medium term. Of the 6 modules, guided participants completed a mean of 3.46 modules (SD 2.25), with 46% (38/82) being compliant (completing ≥4 modules). Similar effect sizes and compliance rates were found for unguided i-RFCBT.

    Conclusions: Guided i-RFCBT can reduce the onset of depression in high-risk young people reporting high levels of worry/rumination and stress. The feasibility study argues for formally testing unguided i-RFCBT for prevention: if the observed effect sizes are robustly replicated in a phase III trial, it has potential as a scalable prevention intervention.

    Trial Registration: ISRCTN Registry ISRCTN12683436; (Archived by WebCite at

    International Registered Report Identifier (IRRID): RR2-10.1186/s13063-015-1128-9

    J Med Internet Res 2019;21(5):e11349





    Depression is the leading cause of disease burden worldwide, accounting for 7.5% of all years lived with disability in 2015 [1], with considerable individual, societal, and economic consequences. Although there are effective evidence-based acute treatments, their impact is limited because of poor access to treatments [2], high rates of nonresponse [3], and the recurrent nature of depression, with 50% to 80% of patients experiencing 2 or more episodes [4]. It is estimated that even with optimal acute treatment at full coverage, only 34% of the disease burden would be averted [5]. As a consequence, a strong case has been made that prevention is needed to reduce the global burden of depression [6].

    Preventive interventions, largely using cognitive behavioral therapy (CBT) approaches, can reduce symptoms of depression and prevent the incidence of depression [7-9], with an average reduction of 21% in incidence rates [10]. These meta-analyses suggest that targeted interventions (selective interventions aimed at subgroups with known risk factors and indicated interventions aimed at those with subclinical symptoms) produce larger and longer-lasting effects than universal interventions aimed at entire populations. A meta-analysis [11] of 21 preventive interventions (15 using CBT approaches) found that selective interventions and indicated interventions had lower incidence rate ratios (IRR; 0.72 and 0.76, respectively) relative to controls than universal interventions (IRR=0.90). Merry et al [8] also found that both universal and targeted interventions reduced incidence relative to no intervention in the short-to-medium term (3 to 9 months postintervention) but only targeted interventions reduced incidence at 12 months. Thus, targeting at-risk groups may improve the efficacy of preventive interventions for depression [6], in part because the base rate is higher in targeted samples, so it is easier to detect a significant effect with smaller sample sizes [12].

    The incidence of depression rises steeply from the age of 14 years through young adulthood, with increased rates in females (2:1, female:male) emerging at around the age of 12 years and continuing into young adulthood [13]. The UK Adult Psychiatric Morbidity Survey found increasing rates of common mental health disorders (CMDs; incorporating depression and anxiety disorders) among young women (aged 16 to 24 years), rising from 22.2% in 2007 to 28.2% in 2014 [14], with rates in young women almost three times those of young men (10.0%) in 2014. As early onset is linked to greater chronicity [15] and other negative long-term outcomes, such as poor academic and occupational performance [1,16], prevention may be particularly effective and impactful for this age group.

    Within this age range, university students are a particularly high-risk group, with a weighted mean prevalence for depression of 30.6% (range 10% to 85%) across 24 studies [17] relative to estimates of 10.8% to 22% in nonstudents of the same age range [18,19]. This increased prevalence may be due to the specific pressures of university and associated lifestyle changes, such as leaving the family home for the first time, forming new friendships, more self-directed learning, and irregular sleep patterns [20]. Students who experience mental health difficulties during their studies are at greater risk of poor academic outcomes [16] and dropout [21].

    Despite these challenges, students often do not seek help from relevant services [22,23]. Alternative delivery modes, such as Web-based interventions, offer advantages that may be attractive to students, including availability at any time and place, anonymity which may reduce the stigma of seeking help, and more time to reflect on the treatment material [12,24,25]. A recent systematic review and meta-analysis of 17 Web-based and computer-delivered interventions for higher education students found reductions in depression, anxiety, and stress when compared with inactive controls [26]. However, sample sizes were generally small, and the authors recommend further larger-scale trials to assess the effectiveness of Web-based interventions in university students.

    One such relatively large-scale trial by Topper et al [27] tested a guided Web-based targeted preventive intervention for 251 high school and university students aged 15 to 22 years with high levels of self-reported worry and rumination. Participants were randomized to face-to-face group rumination-focused cognitive-behavioral therapy (RFCBT), guided Web-based RFCBT (i-RFCBT), or a no-intervention control group. This preventive intervention is based on RFCBT, previously shown to be effective in treating residual depression [28]. There is considerable evidence that rumination plays a causal role in the onset and duration of major depressive episodes (MDEs) [29,30]. Within a student population, rumination predicts change in depression over 6 months [31]. Rumination interacts with other risk factors to both maintain depression (the combination of rumination, low self-esteem, and stressful life events predicts maintenance of depressive symptoms over 6 weeks [32]) and predict the onset of depressive symptoms (engaging in rumination in response to stress prospectively predicted an increase in subsequent depressive symptoms [33]). These studies suggest that specifically targeting ruminative responses to stressful events could reduce depression.

    RFCBT specifically targets repetitive negative thought (RNT), incorporating both rumination and worry, defined as a thinking style that: (1) is repetitive, intrusive, and difficult to disengage from; (2) is perceived as unproductive; and (3) captures mental capacity [34]. RNT is a transdiagnostic process, involved in the onset and maintenance of a range of emotional disorders including depression and anxiety as well as physical health issues [29,35], including in children and adolescents [36-39]. Targeting transdiagnostic risk factors has the potential to improve the efficacy of prevention by impacting on multiple disorders with a single intervention [27].

    In the Topper et al [27] trial, both Web-based and group-delivered RFCBT reduced symptoms of depression and anxiety (d=0.36 to 0.72), relative to controls. Cumulative incidence rates at the final 12-month follow-up were significantly lower in both RFCBT intervention conditions for depression (14.7% Web-based; 15.3% group) relative to the usual care control condition (32.4% depression), with no difference between i-RFCBT versus group RFCBT. In support of the hypothesized mechanism of change, reductions in worry and rumination were found to mediate the effects of the interventions on prevalence of depression and Generalized Anxiety Disorder (GAD). These findings suggest that targeting rumination may have preventive effects for depression and are consistent with evidence that targeted prevention can be effective in adolescents and young adults.


    Topper et al [27] included both secondary and university students to form a heterogeneous sample. Given the evidence that undergraduates may form a distinct at-risk subgroup for depression, the primary aim of this phase III efficacy trial was to test whether these beneficial effects of guided i-RFCBT on onset of depression relative to usual care [27] could be extended to a selective UK high-risk undergraduate population.

    We also aimed to address several key limitations of the Topper et al [27] trial: (1) there was no diagnostic interview to assess depression, and self-report measures were only able to estimate point prevalence caseness and (2) as history of depression was not assessed, participants’ previous history of depression was not known, and therefore it was not possible to discriminate whether the intervention prevented first onset or relapse/recurrence of depression. To address these methodological limitations, we included a well-validated diagnostic interview (Structured Clinical Interview for DSM-IV [SCID-I] [40]) to increase accuracy of the current diagnostic status and measure retrospective incidence.

    We hypothesized that in high-ruminating undergraduates, guided i-RFCBT, relative to usual care, would significantly reduce the onset of MDEs over the course of the 15 months post randomization follow-up (primary outcome). Rumination has also been found to increase the negative effect of stressful life events on depressive symptoms in young people and students [32,33]. This observed interaction is consistent with the evidence that rumination contributes to depression by exacerbating existing negative mood and negative cognitions and by repetitively dwelling on difficulties [29,30], such as results from stressful events. As such, a tendency to ruminate would be expected to have less impact when things are going well and there is less to dwell on, relative to when things are difficult. We therefore hypothesized that i-RFCBT would be particularly beneficial for high-ruminating undergraduates who were also experiencing stressful life events, as this would be the group for whom rumination would be most detrimental.

    As a secondary aim, we explored the feasibility and acceptability of an unguided version of i-RFCBT to prevent depression. Topper et al [27] used i-RFCBT that was guided and supported by a therapist because past evidence suggested that, at least for acute treatment for depression, guided Web-based cognitive behavioral therapy (i-CBT) is significantly more effective than unguided (ie, self-help) i-CBT [41-43], and only guided i-CBT produces similar treatment effects to face-to-face therapy in patients with acute depressive symptoms [24]. A key rationale for Web-based therapy is to increase the coverage, availability, and accessibility of treatment, by potentially reaching large numbers of people through the internet and by overcoming hurdles such as geographical distance, poor mobility, and scheduling appointments during standard office hours. However, any form of guided i-CBT (including i-RFCBT) is necessarily limited in its scalability because coverage is determined by the number and availability of therapists. In contrast, an unguided form of Web-based therapy has nearly limitless scalability as there are no such constraints and, thus, even with smaller effect sizes than guided interventions, has significant potential to reduce the disease burden of depression [44]. Such an intervention would be particularly beneficial for preventing depression because effective prevention requires an intervention to be highly scalable and able to reach very high numbers of people. As a secondary question, we therefore explored the feasibility and acceptability of an unguided version of i-RFCBT in a quasi-phase II pilot arm and estimated its effect sizes to inform a fully powered trial, with regard to incidence rates and symptom levels (descriptives and CIs).


    Trial Design

    Phase III Efficacy Study

    The phase III study consisted of a single (researcher) blind parallel-group randomized controlled trial (RCT), comparing guided i-RFCBT versus a usual care control group. For full details, see the trial protocol paper [45] and Current Controlled Trials ISRCTN12683436.

    Quasi-Phase II Pilot Arm

    To assess the feasibility of unguided i-RFCBT, a separate adjunct arm of unguided i-RFCBT was included as a quasi-phase II pilot arm. For efficiency, participants were randomized to this arm within the overall trial design, but there was no direct comparison between the unguided and guided arms. The unguided arm was compared with the control group to estimate the effect sizes of an unguided version of i-RFCBT for the planning of future efficacy trials.


    Participants were university students resident in the United Kingdom, aged 18 to 24 years, with elevated RNT, defined as scoring above the 75th percentile on at least one measure of worry/rumination: ≥50 on the Penn State Worry Questionnaire (PSWQ; [46]); ≥40 on the Ruminative Response Scale (RRS; [47]), using the same criteria as Topper et al [27]. As a prevention study, participants were excluded if they met the diagnostic criteria for a current (within the past month) MDE. In addition, potential participants were excluded if they reported any of the following: current and significant substance abuse or dependence; current symptoms/diagnosis of psychosis or bipolar disorder; and current psychological therapy or active suicide risk. In line with standard practice, receipt of antidepressant medication was not an exclusion criterion, providing the dose had been stable for at least 1 month.

    Sample Size Calculation and Recruitment

    For the primary question comparing guided i-RFCBT to usual care control, assuming a similar hazard ratio of 0.41 for the guided i-RFCBT versus usual care control [27], 75 participants per arm would provide 0.86 power (2-tailed 5% alpha level) to detect this effect. For change in depressive symptoms, the observed effect size was d=0.51 [27]. A total of 78 participants per arm would be needed for 80% power to detect a similar effect at the 2-tailed 5% alpha level, allowing for 20% follow-up dropout attrition. With no previous evidence for unguided i-RFCBT, the comparison of unguided i-RFCBT to usual care was conducted as a feasibility study as a first step to conducting a fully powered trial of unguided i-RFCBT. As such, no power or sample size calculations were conducted for the unguided arm. We aimed to recruit the same number (n=78) as the other 2 arms.

    The full recruitment procedure is outlined in Cook and Watkins [45]. Briefly, 1834 university departments in the United Kingdom were contacted between November 14, 2013, and December 10, 2014 (1527 contacted twice) and asked to advertise the study. In total, 336 departments confirmed the study was advertised either by email or as a poster. This advertisement contained a link to an open-access screening website. Twitter and Facebook were also used to circulate the advertisement to young people who expressed an interest in the following terms: stress; worry; rumination; mental health; self-esteem; well-being; research; psychology; CBT; and online therapy. In addition, 3 organizations working with young people or in the field of mental health agreed to advertise the study.

    A 2-step procedure identified eligible participants. In the first step, an open-access screening website with conditional automated feedback identified potential participants for further screening by screening in those with elevated RNT (>75th percentile) using shortened versions of the PSWQ (4 items, range 4 to 20, cut-off ≥12) and RRS (5 items, range 5 to 20, cut-off ≥10), as developed by Topper et al [48]. A conservative cut-off of 15 on PHQ-8 [49] excluded individuals likely to be experiencing a current MDE. Eligible participants provided contact details as consent to be contacted for further telephone screening.

    In the second step, a telephone interview consisted of brief screening questions for alcohol and drug use, symptoms of bipolar disorder and psychosis (Psychosis Screening Questionnaire [PSQ]; [50]), assessment of any relevant past or current treatments, and the SCID-I [40] sections on current and past depressive episodes, dysthymia, and any relevant anxiety disorders and eating disorders. As the primary objective was to investigate the prevention of depression, diagnoses of anxiety disorders and eating disorders were recorded but participants meeting the criteria for any of these disorders were not excluded from the study. The consent to interview was obtained verbally and included providing their general practitioner’s (GP) contact details so that appropriate clinical support could be obtained in the event of disclosure of suicidal risk. The interview was audio-recorded, with consent, so that the diagnostic status could be independently checked. The same researcher conducted the baseline and follow-up telephone interview assessments, ensuring continuity of contact between the research team and participants. A total of 254 participants were eligible, of whom 235 returned written informed consent and were randomized to guided i-RFCBT (n=82), unguided i-RFCBT (n=76), or usual care control (n=77). The Consolidated Standards of Reporting Trials diagram (Figure 1) indicates the numbers excluded at baseline for each of the exclusion criteria.

    Figure 1. Consolidated Standards of Reporting Trials flowchart. MDE: major depressive episode; PHQ: Patient Health Questionnaire; RNT: repetitive negative thought; i-RFCBT: Web-based rumination-focused cognitive behavioral therapy.
    View this figure
    Guided Web-Based Rumination-Focused Cognitive Behavioral Therapy

    The guided intervention was an English version of i-RFCBT (called MindReSolve), translated and adapted from the version used by Topper et al [27] to include case examples relevant to university students (see Multimedia Appendix 1). RFCBT differs from standard CBT by seeking to change the process of thinking rather than the content of individual thoughts [51]. RFCBT [52] was developed from theoretical models and experimental findings indicating distinct types of repetitive thought (RT) with different consequences [29]: unconstructive RT involves an abstract, evaluative processing mode focused on the meaning and evaluation of events and difficulties, leading to a range of negative consequences such as poorer problem-solving and greater emotional reactivity, relative to constructive RT, which involves concrete, specific, and action-oriented processing [53]. RFCBT therefore aims to shift participants from an abstract and evaluative style to a concrete, specific, and action-oriented style [29], consistent with evidence that concreteness training reduces depression [54].

    RNT is also theoretically conceptualized as a mental habit acting as a form of avoidance and maintained by negative reinforcement [55]. RFCBT therefore involves counterconditioning the avoidant ruminative response with more helpful coping strategies and approach behaviors [52]. In practice, this involves the functional analysis of rumination to help users spot triggers for rumination, to distinguish between helpful and unhelpful RT, and to counter condition unhelpful RT with more functional responses through the formulation of contingency If-Then plans [52].

    The internet treatment was delivered on the internet platform and software owned, programmed, and hosted by Minddistrict [56], accessed by a research licence purchased from Minddistrict by the research team. The specific content of the i-RFCBT intervention was developed and entered into the platform using its content management system by the research team led by Edward Watkins, using the same key intervention principles and techniques as face-to-face RFCBT as described by Watkins [52], adapted for the internet. i-RFCBT contains the same key components as face-to-face RFCBT [52], split into 6 1-hour modules, each in turn split into 3 or 4 sessions consisting of a single Web page, with 1 to 2 weeks recommended per module for practice of the techniques. The content includes psychoeducation, mood diaries, experiential audio exercises, pictures, and video vignettes of university students talking about their own experiences of the intervention. Modules follow the same basic structure: reflection on the previous module; introduction of a new technique; experiential in-session exercises; and plans for implementation. The specific behavior-change techniques are drawn from the following groups in the Behavior Change Technique (BCT) Taxonomy (v1) [57]: goals and planning (goal setting, action planning, review behavior, and behavioral contract), feedback and monitoring (self-monitoring of behavior and outcomes), shaping knowledge (information about antecedents), natural consequences (information about social and environmental consequences and monitoring of emotional consequences), associations (prompts/cues and associative learning), repetition and substitution (behavioral practice/rehearsal, behavior substitution, and habit formation), antecedents (restructuring the physical and social environment and avoiding/reducing cues for the behavior), and self-belief (mental rehearsal of successful performance, focus on past success, and self-talk). The key strategies include coaching participants to spot warning signs for rumination and worry, and then to make If-Then plans in which an alternative strategy is repeatedly practised in response to the warning signs. These strategies include being more active, slowing things down, breaking tasks down, opposite action, relaxation, concrete thinking, becoming absorbed, self-compassion, and assertiveness.

    The intervention was accessed individually, for free, on a secure, password-protected website. Access was granted using an email link, inviting the participant to set up a personal account and password. The intervention was supported by qualified clinicians who had received additional specific training in RFCBT. This support consisted of asynchronous written feedback provided by the clinician at the end of each module. Feedback served to highlight positive steps and identified areas to focus on in the following module. Feedback was constrained by template responses for each module, faithful to the RFCBT model, which could be adapted to individual participants’ responses. All the content and module order were identical across participants, ensuring treatment fidelity. Each module was self-paced, but the participants were advised to spend 1 to 2 weeks on each and could only access the next module once feedback from the clinician was received, typically within 2 working days. Clinicians monitored log-ons and sent personalized reminder emails if there was no log-on for over a week. The platform also sent an automatic weekly reminder if the platform had not been accessed for a week. Suicidal risk was also monitored using a well-established departmental protocol to determine the level of risk and seek clinical support as appropriate.

    Therapists were provided with regular supervision with the developer of RFCBT (EW) to further encourage treatment fidelity. All (100%) feedback reviewed by EW were faithful to the intervention model (over 10% of therapist feedback sampled—a minimum of the 3 initial feedback for each therapist, plus a random subset of later feedback).

    Unguided Web-Based Rumination-Focused Cognitive Behavioral Therapy

    The content of unguided i-RFCBT was almost identical to guided i-RFCBT with minor adaptations for self-help to include some automatic Web-based conditional feedback addressing common challenges with the exercises. Access was granted via an email link to set up a personal account and password. Participants could then access all modules without restriction but were advised to spend 1 to 2 weeks on each to allow time for practice. Responses were not monitored except for weekly checks of questionnaires to identify and follow up suicidal risk as necessary.

    Usual Care Control Condition

    Participants in the usual care control condition were permitted to access any other treatments during the study, as necessary. They were also offered access to unguided i-RFCBT at the end of the follow-up period.


    All measures were completed at baseline, 3 months, 6 months, and 15 months unless otherwise stated. Diagnostic interviews for the primary outcome were conducted by telephone, with the option to complete self-report questionnaire measures for secondary outcomes during the telephone interview or request for them to be returned by email/post.

    TheSCID-I [40] is a semistructured diagnostic interview for Axis I DSM-IV diagnoses. The SCID-I was used to assess MDE (current and past), anxiety disorders, and eating disorders. The interrater reliability for Axis I diagnoses is fair to excellent, with a mean Kappa of 0.71 [58]. In the event of disclosure of suicidal risk during the diagnostic interview, the researcher followed a well-established departmental protocol to assess risk and obtain clinical support as needed.

    The Episodic Life Event Interview, part of the University of California Los Angeles Life Stress Interview [59], assessed the number and impact of stressful events since the previous assessment (for the previous 3 months at baseline). Participants provided a list of events experienced and a subjective rating of stress experienced as a result of the worst event. The original scale ranges from 1 none to 5 severe. Participants scored 0 if no events were experienced. To aid analysis and interpretation, stress was recoded to collapse 0 no event and 1 event experienced but no stress into a single no stress category. The recoded stress scale therefore ranges from 0 no stress to 4 severe stress.

    The PSWQ [46] is a 16-item self-report questionnaire assessing frequency, intensity, and automaticity of worry (eg, “My worries overwhelm me” and “I know I shouldn’t worry about things, but I just can’t help it”). It is scored from 1 (not at all typical of me) to 5 (very typical of me), with higher scores indicating higher levels of worry. The internal consistency is high with good test-retest reliability [46]. The PSWQ has also been shown to have good predictive validity for symptoms of anxiety and depression [60].

    The RRS [47] is a self-report measure of frequency of ruminative responses to depressed mood, with items relating to the self (eg, “Think about all your shortcomings, failings, faults and mistakes”), one’s symptoms (eg, “Think about how hard it is to concentrate”) and possible causes and consequences of one’s mood (eg, “Go away by yourself and think about why you feel this way”). Items are scored from 1 (almost never) to 4 (almost always). Higher scores indicate higher levels of rumination. The RRS has good internal consistency, moderate test-retest reliability, acceptable convergent validity, and good predictive validity [47,61,62].

    The Patient Health Questionnaire (PHQ-9; [63]) is a 9-symptom measure of depressive symptoms. Scores range from 0 to 27, with higher scores indicating greater severity. The PHQ-9 is a reliable and valid measure of severity of depressive symptoms [63].

    The GAD Screener (GAD-7; [64]) is a standardized self-report measure of symptoms of anxiety. Scores range from 0 to 21, and higher scores indicate more severe symptoms. Spitzer et al [64] demonstrated good validity and reliability of the GAD-7.

    Demographics and Treatment

    At baseline, participants were asked if they had any family history of depression (including whom and how recently) and whether they had experienced any physical, sexual, or emotional abuse before the age of 16 years (yes/no questions with no further details requested). Participants were asked to report whether they had received any mental health treatments (medication, therapies, and use of self-help materials) before or during the trial. Timing, duration, and dosage (for medication) were recorded.

    Randomization, Allocation Concealment, and Blinding

    Independent computer-generated block randomization (block size of 3), stratified by sex (male vs female) and a history of depression (presence or absence of past depressive episodes), was used to allocate participants to the guided i-RFCBT, unguided i-RFCBT, or usual care control in a 1:1:1 ratio. Varying block sizes were not used as the 2 levels of stratification ensured it would be difficult for the researcher to anticipate or determine allocation. A third party not involved in assessing or treating the participants implemented the random allocation sequence and informed the therapist of the condition for each participant. The researcher responsible for recruitment and screening was blind to allocation and unable to influence the order of consents. As a single blind trial, the researcher conducting outcome assessments was blind to allocation. The researcher was not involved with any element of treatment delivery. To preserve researcher blinding, participants were notified of their treatment allocation by a trial therapist. Owing to the nature of the intervention, participants and therapists could not be blinded.

    Statistical Analysis for the Phase III Efficacy Trial

    Data cleaning followed the protocol set out by Tabachnick and Fidell [65]. Unplanned missing data were handled via multiple imputation (MI). A sensitivity analysis, assuming a variety of MI models (Missing at Random and Missing Not at Random), verified the likely impact of missing data. Auxiliary variables were used to improve the estimation of missing data. Primary analyses were conducted on the intention-to-treat (ITT) sample. Additional analyses assessed the effect of compliance using the Complier Average Causal Effect (CACE) analysis [66]. The CACE analysis provides an unbiased estimate of the benefits of compliance by comparing the compliers in the intervention group to a comparable subgroup of the control group who would have complied had they been offered the intervention. Compliance was defined in the protocol as fully completing at least 4 of 6 modules, that is, accessing all of the sections in each of those modules [45]. Analyses were carried out using statistical software, Stata (StataCorp; version 15.1 [67]).

    As a prevention study, the primary outcome was the occurrence and time to onset of any depressive episode by 15-month follow-up. To investigate this, Cox proportional hazard models were fitted to the depression event data, with the diagnosis of an episode of major depression at any point during the follow-up period as the outcome and time to onset measured in weeks from the randomization date. Participants were censored upon measurement dropout or end of study. The Cox proportional hazard model was initially adjusted for both stratification variables: past depression and gender, as they have previously been found to influence likelihood of depression. In addition, as baseline stress was expected to increase the risk of depression and Topper et al [27] controlled for stressful life events, severity of baseline stress was included in the model. To examine the hypothesis that i-RFCBT would be especially beneficial in high ruminators experiencing high stress, we further tested the potential interactions between intervention condition and baseline stress, and intervention condition and history of depression within the Cox proportional hazard analysis.

    Secondary outcomes of symptom severity and levels of rumination/worry were examined using mixed model analyses of covariance (ANCOVAs): between group (ITT/CACE) and repeated measures (3- to 15-month follow-ups), controlling for baseline symptom levels.

    Feasibility and Acceptability (Quasi-Phase II Pilot Arm)

    Feasibility of data collection procedures was assessed by measuring the missing items on clinical outcome measures, number and timing of dropouts, and whether these varied across arms. The acceptability of the intervention was assessed using a behavioral index, measuring the number of Web-based modules completed.

    Ethical Approval and Informed Consent

    Ethical and professional guidelines were followed at all times, in line with Good Clinical Practice guidelines. Ethical approval was obtained from the Ethics Committee of the School of Psychology, University of Exeter (Ref: 2012/554). Participants returned written informed consent including permission to contact their GP if significant risk was disclosed (see Multimedia Appendix 2).



    For brevity, baseline demographics for the 3 arms are included in Table 1. As noted earlier, the primary comparison is guided i-RFCBT versus usual care control, with a separate analysis of the feasibility and acceptability of the adjunct unguided i-RFCBT arm.

    Survival Analysis: Guided Web-Based Rumination-Focused Cognitive Behavioral Therapy Versus Usual Care Control

    A total of 27 participants in the primary comparison of guided i-RFCBT versus the control completed no follow-ups, and no minimum survival time could be estimated, so the ITT survival analyses were conducted on n=132 (guided n=63 and control n=69). Participants with a family history of depression were more likely to be lost to follow-up than those without: χ21=3.89; P=.049. No other baseline variables were linked to loss to follow-up: all t values on continuous measures were <1.70; all chi-square test values on categorical variables were <1.74; all P values were >.09.

    There was no overall difference in incidence of depression (P=.64): 29% (n=18) of participants receiving guided i-RFCBT and 33% (n=23) of participants receiving usual care experienced an MDE during the follow-up period. A Cox proportional hazard model was conducted, including past depression, gender, and baseline stress as potential predictors of incidence of depression. As the majority (83%) of participants were female and there was no significant effect of gender in predicting depression, this variable was removed from the model, such that the final model controlled for past depression and baseline stress. As expected, history of depression significantly increased risk, with participants with a history of depression over two and a half times more likely to experience an MDE than participants without: hazard ratio (HR) 2.62, 95% CI 1.37 to 5.01; P=.004. Baseline stress marginally increased the risk of MDEs: HR 1.40, 95% CI 0.99 to 1.99; P=.06. When controlling for both past depression and baseline stress, there was a 34% reduced risk of depression in the guided i-RFCBT condition relative to usual care, although this difference was not significant: HR 0.66, 95% CI 0.35 to 1.25; P=.20 (see Figure 2).

    For Cox proportional hazard models including the interactions between intervention condition and baseline stress, and intervention condition and history of depression, there was no differential effect of intervention between first onset (ie, no history of depression) or relapse/recurrence (ie, past history of depression) for incidence of major depression, and this interaction was removed from the final model (HR 0.54, 95% CI 0.15 to 1.94; P=.34; guided i-RFCBT: 38.9% first onset; 61.1% relapse vs usual care: 36.4% first onset; 63.6% relapse). Both the effects of past depression (HR 2.52, 95% CI 1.32 to 4.81; P=.005) and baseline stress (HR 1.99, 95% CI 1.22 to 3.24; P=.006) remained significant. As hypothesized, there was a significant interaction of the intervention condition by baseline stress (HR 0.43, 95% CI 0.21 to 0.87; P=.02), indicating a greater benefit of guided i-RFCBT relative to usual care (risk of MDEs decreased by 57%) for undergraduates with higher baseline stress.

    Table 1. Baseline characteristics of usual care, guided Web-based rumination-focused cognitive behavioral therapy (i-RFCBT), and unguided i-RFCBT intention-to-treat samples.
    View this table
    Figure 2. Survival curves for guided Web-based rumination-focused cognitive behavioral therapy (i-RFCBT) and usual care controls, adjusted for past depression and baseline stress.
    View this figure

    Plotting the interaction between the intervention group and baseline stress (see Figure 3) suggests that at higher levels of stress, guided i-RFCBT markedly reduces the risk of a depressive episode relative to usual care, with this effect reversing at low levels of stress (albeit in a small number of participants, only 13.1% scoring either 0 or 1).

    As further sensitivity analyses, to investigate the effect of compliance on outcomes, we conducted a CACE analysis, using the Loeys and Goetghebur [66] method, which only allows for inclusion of the randomization variable in the model, and using regression-based adjustments to include past depression and baseline stress, which compares compliers in the intervention group to all other participants [68]. The mean completion for guided i-RFCBT was 3.46 (SD 2.25) for the full ITT sample (n=159), with 46% (38/82) being compliant by completing at least 4 of the 6 modules. The rates of compliance were higher among those with follow-up outcome data (n=132) as used for the CACE analysis at 60% (38/63). The results of the CACE analyses (see Multimedia Appendix 3) were equivalent to the ITT analysis. We therefore have only reported the primary ITT analysis.

    Figure 3. Survival curves for guided Web-based rumination-focused cognitive behavioral therapy (i-RFCBT), unguided i-RFCBT, and usual care controls at each level of baseline stress.
    View this figure

    Secondary Analyses on Patient Health Questionnaire, Generalized Anxiety Disorder, Ruminative Response Scale, and Penn State Worry Questionnaire

    Baseline adjusted ANCOVAs were conducted for each of the symptom measures, at each of the 3 follow-ups. Estimated means, between-group differences, and CIs are displayed in Table 2 for the case completers and following MIs (50 imputations). For the complete cases, at 3 months, rumination scores were significantly lower for guided i-RFCBT relative to usual care; at 6 months, both worry and depression scores were significantly lower for guided i-RFCBT relative to usual care, and there was no evidence of significant between-group differences at 15 months. Similar patterns were found when using MI to account for differing levels of missing data across the follow-ups.

    Table 2. Baseline adjusted symptom measures at 3, 6, and 15 months: guided Web-based rumination-focused cognitive behavioral therapy versus usual care controls.
    View this table

    Retention, Acceptability and Effect Sizes of Unguided Web-Based Rumination-Focused Cognitive Behavioral Therapy

    A total of 19 (25%) unguided participants did not complete any follow-up assessments. Participants were significantly more likely to be lost to follow-up in unguided i-RFCBT than in usual care: χ21=4.53; P=.03.

    Owing to the exploratory nature of the unguided version of i-RFCBT, no formal CACE analysis of compliance was undertaken for the unguided intervention. In the full ITT sample, unguided participants completed an average of 2.66 modules (SD 2.35). The rates of compliance (38% unguided) were not significantly different from guided i-RFCBT (χ21=1.08; P=.30). For the unguided intervention, participants logged in an average of 6.25 times (SD 5.21) and accessed the intervention over an average period of 114.92 days (SD 105.51). The median (interquartile range) was 87 days (22-173). Guided participants logged in an average of 7.97 times (SD 5.65) and accessed the intervention over an average period of 110.13 days (SD 108.44). The median (interquartile range) was 67 days (38-156).

    Estimates of Hazard Ratios for Unguided Web-Based Rumination-Focused Cognitive Behavioral Therapy Versus Usual Care

    No formal significance analyses were undertaken, but hazard ratios and CIs were estimated relative to usual care. Using a Cox proportional hazard model including past depression and baseline stress, unguided i-RFCBT showed a 36% reduced risk of developing a depressive episode relative to controls: HR 0.64, 95% CI 0.33 to 1.24. A similar interaction between the intervention and baseline stress was found as for guided i-RFCBT (HR 0.48, 95% CI 0.23 to 1.00), such that unguided i-RFCBT had larger effect sizes for undergraduates with moderate-to-severe levels of baseline stress (see Figure 3).

    Table 3. Baseline adjusted symptom measures at 3, 6, and 15 months: unguided Web-based rumination-focused cognitive behavioral therapy versus usual care controls.
    View this table

    Between-group differences for unguided i-RFCBT versus usual care were estimated with baseline adjusted ANCOVAs for both case completers and using multiple imputations (50 imputations). Estimated means and CIs are displayed in Table 3. Owing to the exploratory nature of this comparison, significance testing was not conducted. Patterns of change and CIs indicate similar symptom changes to those found in the guided i-RFCBT versus usual care control ANCOVAs.


    Principal Findings and Comparison With Previous Work

    The main aim was to test if guided i-RFCBT could be effective in preventing depression in undergraduate students in the United Kingdom over 1-year follow-up. When controlling for both past depression and baseline stress, guided i-RFCBT reduced the risk of experiencing an MDE by 34% relative to usual care. Although this effect size was not significant and smaller than that found by Topper et al [27], it is consistent with the wider prevention literature, which reports an average reduction in incidence of 21% [10] and a 28% (IRR=.72) reduction in incidence relative to controls for selective, predominantly CBT, interventions [11]. It may be that this study was underpowered to detect a main preventive effect of i-RFCBT as it used a larger effect size estimate derived from Topper et al [27].

    As hypothesized, guided i-RFCBT was significant at preventing the onset of MDEs in high-risk undergraduates relative to usual care when they experienced moderate or above levels of baseline stress, with a hazard ratio of 0.43 when moderated by baseline stress. This is consistent with theoretical models of rumination and the RFCBT treatment approach. The tendency to ruminate about difficulties or low mood is more likely to increase the risk for depression in the context of stressful events, which activates that habitual tendency and provides subject matter to ruminate about. Even someone with a habitual tendency to ruminate is less likely to have frequent rumination in the absence of any difficulties. Furthermore, one key mechanism by which rumination is proposed to increase vulnerability to depression is by exacerbating and prolonging negative affect and distress [29,30]: rumination does not have deleterious effects in the absence of negative mood, and it is thus the confluence of stressful events that lower mood and the tendency to ruminate that particularly confer the risk for depression [32]. This pattern of results suggests a partial replication of Topper et al [27], by indicating that guided i-RFCBT may be a helpful preventive intervention for university students with high levels of rumination and worry, who also experience at least moderate levels of stress.

    We note that the observed interaction between i-RFCBT and baseline stress could also be interpreted as indicating that RFCBT is unhelpful compared with usual care, for users who are experiencing little to no current stress. However, given the small number of participants who reported low levels of baseline stress, this reversal is based on low power and needs to be treated with caution.

    The findings on the symptom measures suggest that guided i-RFCBT was effective in the short-to-medium term, by reducing rumination, worry, and symptoms of depression at 3 and 6 months relative to usual care, but that these improvements were not sustained over the longer term. Watkins and Nolen-Hoeksema [55] hypothesized that rumination could be conceptualized as a learnt habit, triggered by particular cues such as low mood. Within this analysis, successful long-term reduction of the ruminative habit requires extensive repetition and rehearsal of alternative more adaptive responses to the triggers for rumination. It may be that i-RFCBT was too brief or that participants did not practise enough to produce long-term change in the ruminative habit. It may also be that further engagement and booster sessions some months after the initial intervention phase would enhance the longer-term effects of the intervention [69]. These could take the form of explicit reminders to practice techniques (flashcards and text/email reminders; [70]) or increasing the generalizability of the new, more helpful techniques across a broader range of contexts [70].

    One possible reason for the difference in findings between this study and Topper et al [27] is the means of assessing onset of depression: Reducing Stress and Preventing Depression (RESPOND) used structured clinical interviews, whereas Topper et al [27] used cut-offs on self-report questionnaires, which may overestimate incidence. Another potential explanation is the different samples. Although their sample included university students, the average age was 17.5 years, compared to 20.4 years in RESPOND. Cases of depression begin to rise steeply from the age of 14 [13], so it may be that the developmental risks during mid to late adolescence differ from those in university students and either that i-RFCBT was more efficacious in younger participants or the base rate was higher in the younger sample, increasing the power of the trial.

    Compliance rates and the pattern of findings and preliminary effect sizes and CIs for unguided i-RFCBT were similar to those for guided i-RFCBT. These findings are in contrast to the literature on Web-based acute treatment for depression, which generally demonstrates larger effect sizes for guided interventions relative to unguided [41-43]. This benefit of therapist guidance has also previously been found for indicated preventive interventions in university students [71]. As a preliminary finding from a feasibility pilot, further large-scale trials are needed to confirm whether this potential equivalence between unguided and guided i-RFCBT is robust. We speculate that only selecting high-ruminating participants for the trial meant that i-RFCBT was highly relevant and engaging to participants, thus ameliorating the relative benefits of guidance on treatment motivation and completion.

    Given the need for widespread dissemination of preventive interventions, an efficacious unguided intervention would be valuable, even if it had somewhat reduced effect sizes relative to the guided version, because it would not be constrained by therapist numbers or availability unlike a guided treatment and could be enormously scaled up to increase accessibility [44]. In addition, unguided interventions may benefit a previously unreached population as many university students do not seek professional help for mental health difficulties [22,23] and could therefore be more attracted to self-help interventions. In support of this, an unguided preventive intervention for students with elevated distress ratings reduced depressive symptoms relative to usual care at 2 months follow-up [72], with two-thirds of the trial completers reporting an unmet need (needing help but not seeking it) in the previous year. As i-RFCBT targets worry and rumination, rather than focusing on depression, this may further attract those who prefer self-help to manage their symptoms as worry is a common experience without the perceived stigma of mental illness [27]. These initial findings on the acceptability and effect sizes of the unguided version provide some promise in terms of potential benefits and suggests the value of further studies to formally test unguided i-RFCBT as a preventive intervention.

    Despite the need for larger-scale trials to test the robustness of these findings, several strengths of RESPOND are identified. First, the RESPOND trial addressed some of the methodological limitations of the Topper et al [27] trial by including diagnostic interviewing. This allowed for retrospective diagnoses, capturing any episodes occurring between follow-up interviews, as well as baseline history of depression to assess the effect of previous history on risk of a further MDE.

    The use of Web-based and telephone-based measures allowed for recruitment throughout the United Kingdom, with participants from a wide range of university departments and geographical locations, increasing the generalizability of the findings within this demographic. The target sample size was achieved through this recruitment strategy and this would therefore be a suitable approach for a larger scale trial of i-RFCBT.


    There were several limitations to the study. First, the sample was disproportionately female, limiting the generalizability of the findings. However, females consistently report higher levels of rumination [73] and higher levels of depression, so a trial selecting on this basis will necessarily attract more female participants.

    Second, despite a successful recruitment strategy, there was a considerable proportion of missing data at follow-up, particularly in the intervention conditions (albeit in the context of planning the sample size for 20% drop-out attrition). In addition, follow-up assessments were sometimes incomplete as participants did not always return the questionnaires after the follow-up interview, despite reminders being sent. Future trials should therefore further emphasize to participants the importance of follow-up data during the baseline assessment and ensure all measures are completed during the interview. Finally, common to many electronic mental health trials, the participants were not blind to the treatment condition, and, as such, the results could have been influenced by response bias and expectancy effects.


    Despite these limitations, taken together, the findings from the Topper et al [27] trial and from this RESPOND trial suggest that i-RFCBT is an effective and acceptable intervention for preventing depression in adolescents and undergraduates experiencing high levels of rumination and worry. This demonstrates the value in targeting a preventive intervention at identified risk factors. This intervention may be particularly effective in individuals experiencing high levels of stress. The initial findings relating to unguided i-RFCBT suggest this may be efficacious in preventing depression, which, if shown to be robust in a fully powered trial, would have significant implications for the scalability of i-RFCBT.


    This work was supported by the University of Exeter, United Kingdom, through a matched funded PhD studentship to a successful Wellcome Trust Capital Bid. The funders had no involvement in the research or the submission of this manuscript for publication.

    The authors are grateful to Jenny Cadman, Dr Yusuke Umegaki (therapists for guided therapy and assisting with randomization), and Rosario Melero (assisting with recruitment and therapist for guided therapy).

    Conflicts of Interest

    EW developed the original RFCBT intervention. The other authors declare they have no competing interests.

    Multimedia Appendix 1

    Screenshots of Web-based rumination-focused cognitive behavioral therapy.

    PDF File (Adobe PDF File), 766KB

    Multimedia Appendix 2

    Information sheet and consent form.

    PDF File (Adobe PDF File), 79KB

    Multimedia Appendix 3

    Complier average causal effect analysis.

    PDF File (Adobe PDF File), 41KB

    Multimedia Appendix 4

    CONSORT-EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 524KB


    1. World Health Organization. Geneva, Switzerland: WHO; 2017. Depression and other common mental disorders: global health estimates   URL: [accessed 2018-04-23] [WebCite Cache]
    2. Eisenberg D, Hunt J, Speer N, Zivin K. Mental health service utilization among college students in the United States. J Nerv Ment Dis 2011 May;199(5):301-308. [CrossRef] [Medline]
    3. Griffiths CA, Griffiths LJ. Recovery and reliable change rates for patients scoring severe on depression, anxiety or impaired functioning in a psychological therapies service: IAPT. Ment Health Rev J 2015 Mar 9;20(1):28-35. [CrossRef]
    4. Judd LL. The clinical course of unipolar major depressive disorders. Arch Gen Psychiatry 1997 Nov;54(11):989-991. [Medline]
    5. Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H. Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders. Br J Psychiatry 2004 Jun;184:526-533 [FREE Full text] [Medline]
    6. Topper M, Emmelkamp P, Ehring T. Improving prevention of depression and anxiety disorders: Repetitive negative thinking as a promising target. Appl Prev Psychol 2010 Jun;14(1-4):57-71. [CrossRef]
    7. Horowitz JL, Garber J. The prevention of depressive symptoms in children and adolescents: a meta-analytic review. J Consult Clin Psychol 2006 Jun;74(3):401-415. [CrossRef] [Medline]
    8. Merry SN, Hetrick SE, Cox GR, Brudevold-Iversen T, Bir JJ, McDowell H. Psychological and educational interventions for preventing depression in children and adolescents. Cochrane Database Syst Rev 2011 Dec 7(12):CD003380. [CrossRef] [Medline]
    9. Stice E, Shaw H, Bohon C, Marti CN, Rohde P. A meta-analytic review of depression prevention programs for children and adolescents: factors that predict magnitude of intervention effects. J Consult Clin Psychol 2009 Jun;77(3):486-503 [FREE Full text] [CrossRef] [Medline]
    10. van Zoonen K, Buntrock C, Ebert DD, Smit F, Reynolds CF, Beekman AT, et al. Preventing the onset of major depressive disorder: a meta-analytic review of psychological interventions. Int J Epidemiol 2014 Apr;43(2):318-329 [FREE Full text] [CrossRef] [Medline]
    11. Cuijpers P, van Straten A, Smit F, Mihalopoulos C, Beekman A. Preventing the onset of depressive disorders: a meta-analytic review of psychological interventions. Am J Psychiatry 2008 Oct;165(10):1272-1280. [CrossRef] [Medline]
    12. Muñoz RF, Cuijpers P, Smit F, Barrera AZ, Leykin Y. Prevention of major depression. Annu Rev Clin Psychol 2010;6:181-212. [CrossRef] [Medline]
    13. Hankin BL, Young JF, Abela JR, Smolen A, Jenness JL, Gulley LD, et al. Depression from childhood into late adolescence: influence of gender, development, genetic susceptibility, and peer stress. J Abnorm Psychol 2015 Nov;124(4):803-816 [FREE Full text] [CrossRef] [Medline]
    14. McManus S, Bebbington P, Jenkins R, Brugha T. National Health Service. Leeds, UK: NHS Digital; 2016. Mental health and wellbeing in England: Adult Psychiatric Morbidity Survey 2014   URL: [accessed 2018-04-23] [WebCite Cache]
    15. Pine DS, Cohen P, Gurley D, Brook J, Ma Y. The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Arch Gen Psychiatry 1998 Jan;55(1):56-64. [Medline]
    16. Hysenbegasi A, Hass SL, Rowland CR. The impact of depression on the academic productivity of university students. J Ment Health Policy Econ 2005 Sep;8(3):145-151. [Medline]
    17. Ibrahim AK, Kelly SJ, Adams CE, Glazebrook C. A systematic review of studies of depression prevalence in university students. J Psychiatr Res 2013 Mar;47(3):391-400. [CrossRef] [Medline]
    18. Aalto-Setälä T, Marttunen M, Tuulio-Henriksson A, Poikolainen K, Lönnqvist J. One-month prevalence of depression and other DSM-IV disorders among young adults. Psychol Med 2001 Jul;31(5):791-801. [Medline]
    19. Newman DL, Moffitt TE, Caspi A, Magdol L, Silva PA, Stanton WR. Psychiatric disorder in a birth cohort of young adults: prevalence, comorbidity, clinical significance, and new case incidence from ages 11 to 21. J Consult Clin Psychol 1996 Jun;64(3):552-562. [Medline]
    20. National Institute of Mental Health. Bethesda, MD: NIMH; 2003. Depression and college students   URL: [accessed 2018-04-23] [WebCite Cache]
    21. Eisenberg D, Golberstein E, Hunt JB. Mental health and academic success in college. B E J Econom Anal Policy 2009;9(1):40. [CrossRef]
    22. Eisenberg D, Gollust SE, Golberstein E, Hefner JL. Prevalence and correlates of depression, anxiety, and suicidality among university students. Am J Orthopsychiatry 2007 Oct;77(4):534-542. [CrossRef] [Medline]
    23. National Union of Students. Lost in transition? – provision of mental health support for 16-21 year olds moving to further and higher education. 2015 Presented at: All Party Parliamentary Group on Students; December 14, 2015; London, UK   URL: https:/​/www.​​PageFiles/​2161132/​APPG%20on%20Students%20-%20December%20-%20Speaker%20briefing%20-%20Mental%20Health%20(002).​pdf
    24. Andersson G. The promise and pitfalls of the internet for cognitive behavioral therapy. BMC Med 2010;8:82 [FREE Full text] [CrossRef] [Medline]
    25. Richardson T, Stallard P, Velleman S. Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: a systematic review. Clin Child Fam Psychol Rev 2010 Sep;13(3):275-290. [CrossRef] [Medline]
    26. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res 2014;16(5):e130 [FREE Full text] [CrossRef] [Medline]
    27. Topper M, Emmelkamp PM, Watkins E, Ehring T. Prevention of anxiety disorders and depression by targeting excessive worry and rumination in adolescents and young adults: a randomized controlled trial. Behav Res Ther 2017 Dec;90:123-136. [CrossRef] [Medline]
    28. Watkins ER, Mullan E, Wingrove J, Rimes K, Steiner H, Bathurst N, et al. Rumination-focused cognitive-behavioural therapy for residual depression: phase II randomised controlled trial. Br J Psychiatry 2011 Oct;199(4):317-322 [FREE Full text] [CrossRef] [Medline]
    29. Watkins ER. Constructive and unconstructive repetitive thought. Psychol Bull 2008 Mar;134(2):163-206 [FREE Full text] [CrossRef] [Medline]
    30. Nolen-Hoeksema S, Wisco BE, Lyubomirsky S. Rethinking rumination. Perspect Psychol Sci 2008 Sep;3(5):400-424. [CrossRef] [Medline]
    31. Morrison R, O'Connor RC. Predicting psychological distress in college students: the role of rumination and stress. J Clin Psychol 2005 Apr;61(4):447-460. [CrossRef] [Medline]
    32. Ciesla JA, Felton JW, Roberts JE. Testing the cognitive catalyst model of depression: does rumination amplify the impact of cognitive diatheses in response to stress? Cogn Emot 2011 Dec;25(8):1349-1357 [FREE Full text] [CrossRef] [Medline]
    33. Connolly SL, Alloy LB. Rumination interacts with life stress to predict depressive symptoms: an ecological momentary assessment study. Behav Res Ther 2017 Oct;97:86-95. [CrossRef] [Medline]
    34. Ehring T, Zetsche U, Weidacker K, Wahl K, Schönfeld S, Ehlers A. The Perseverative Thinking Questionnaire (PTQ): validation of a content-independent measure of repetitive negative thinking. J Behav Ther Exp Psychiatry 2011 Jun;42(2):225-232 [FREE Full text] [CrossRef] [Medline]
    35. Nolen-Hoeksema S, Watkins ER. A heuristic for developing transdiagnostic models of psychopathology: explaining multifinality and divergent trajectories. Perspect Psychol Sci 2011 Nov;6(6):589-609. [CrossRef] [Medline]
    36. Abela JRZ, Brozina K, Haigh EP. An examination of the response styles theory of depression in third- and seventh-grade children: a short-term longitudinal study. J Abnorm Child Psychol 2002 Oct;30(5):515-527. [Medline]
    37. McLaughlin KA, Hatzenbuehler ML. Mechanisms linking stressful life events and mental health problems in a prospective, community-based sample of adolescents. J Adolesc Health 2009 Feb;44(2):153-160 [FREE Full text] [CrossRef] [Medline]
    38. Nolen-Hoeksema S, Stice E, Wade E, Bohon C. Reciprocal relations between rumination and bulimic, substance abuse, and depressive symptoms in female adolescents. J Abnorm Psychol 2007 Feb;116(1):198-207. [CrossRef] [Medline]
    39. Schwartz JA, Koenig LJ. Response styles and negative affect among adolescents. Cogn Ther Res 1996 Feb;20(1):13-36. [CrossRef]
    40. First M, Spitzer R, Gibbon M, Williams J. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition With Psychotic Screen (SCID-I/P W/ PSY SCREEN). New York: Biometrics Research, New York State Psychiatric Institute; 2002.
    41. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205. [CrossRef] [Medline]
    42. Gellatly J, Bower P, Hennessy S, Richards D, Gilbody S, Lovell K. What makes self-help interventions effective in the management of depressive symptoms? Meta-analysis and meta-regression. Psychol Med 2007 Sep;37(9):1217-1228. [CrossRef] [Medline]
    43. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
    44. Cuijpers P, Riper H, Andersson G. Internet-based treatment of depression. Curr Opin Psychol 2015 Aug;4:131-135. [CrossRef]
    45. Cook L, Watkins E. Guided, internet-based, rumination-focused cognitive behavioural therapy (i-RFCBT) versus a no-intervention control to prevent depression in high-ruminating young adults, along with an adjunct assessment of the feasibility of unguided i-RFCBT, in the REducing Stress and Preventing Depression trial (RESPOND): study protocol for a phase III randomised controlled trial. Trials 2016 Jan 04;17:1 [FREE Full text] [CrossRef] [Medline]
    46. Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behav Res Ther 1990;28(6):487-495. [Medline]
    47. Nolen-Hoeksema S, Morrow J. A prospective study of depression and posttraumatic stress symptoms after a natural disaster: the 1989 Loma Prieta Earthquake. J Pers Soc Psychol 1991 Jul;61(1):115-121. [Medline]
    48. Topper M, Emmelkamp PM, Watkins E, Ehring T. Development and assessment of brief versions of the Penn State Worry Questionnaire and the Ruminative Response Scale. Br J Clin Psychol 2014 Nov;53(4):402-421. [CrossRef] [Medline]
    49. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009 Apr;114(1-3):163-173. [CrossRef] [Medline]
    50. Bebbington PE, Nyani T. The Psychosis Screening Questionnaire. Int J Methods Psychiatr Res 1995;5:11-19 [FREE Full text]
    51. Watkins E, Scott J, Wingrove J, Rimes K, Bathurst N, Steiner H, et al. Rumination-focused cognitive behaviour therapy for residual depression: a case series. Behav Res Ther 2007 Sep;45(9):2144-2154. [CrossRef] [Medline]
    52. Watkins E. Rumination-Focused Cognitive-Behavioral Therapy for Depression. New York: The Guilford Press; 2016.
    53. Watkins E, Moberly NJ, Moulds ML. Processing mode causally influences emotional reactivity: distinct effects of abstract versus concrete construal on emotional response. Emotion 2008 Jun;8(3):364-378 [FREE Full text] [CrossRef] [Medline]
    54. Watkins ER, Taylor RS, Byng R, Baeyens C, Read R, Pearson K, et al. Guided self-help concreteness training as an intervention for major depression in primary care: a Phase II randomized controlled trial. Psychol Med 2012 Jul;42(7):1359-1371 [FREE Full text] [CrossRef] [Medline]
    55. Watkins ER, Nolen-Hoeksema S. A habit-goal framework of depressive rumination. J Abnorm Psychol 2014 Feb;123(1):24-34. [CrossRef] [Medline]
    56. Minddistrict.   URL: [accessed 2018-06-19] [WebCite Cache]
    57. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95. [CrossRef] [Medline]
    58. Lobbestael J, Leurgans M, Arntz A. Inter-rater reliability of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID I) and Axis II Disorders (SCID II). Clin Psychol Psychother 2011;18(1):75-79. [CrossRef] [Medline]
    59. Hammen C. Episodic Life Event Interview: UCLA Life Stress Interview. Los Angeles, CA: UCLA Department of Psychology; 2004.
    60. Hong RY. Worry and rumination: differential associations with anxious and depressive symptoms and coping behavior. Behav Res Ther 2007 Feb;45(2):277-290. [CrossRef] [Medline]
    61. Butler L, Nolen-Hoeksema S. Gender differences in responses to depressed mood in a college sample. Sex Roles 1994 Mar;30(5-6):331-346. [CrossRef]
    62. Nolen-Hoeksema S. The role of rumination in depressive disorders and mixed anxiety/depressive symptoms. J Abnorm Psychol 2000 Aug;109(3):504-511. [Medline]
    63. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline]
    64. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    65. Tabachnick B, Fidell L. Using Multivariate Statistics (4th Edition). Needham Heights, MA: Allyn & Bacon; 2001.
    66. Loeys T, Goetghebeur E. A causal proportional hazards estimator for the effect of treatment actually received in a randomized trial with all-or-nothing compliance. Biometrics 2003 Mar;59(1):100-105. [Medline]
    67. StataCorp. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC; 2017.
    68. Odondi L, McNamee R. Performance of statistical methods for analysing survival data in the presence of non-random compliance. Stat Med 2010 Dec 20;29(29):2994-3003. [CrossRef] [Medline]
    69. van Voorhees BW, Mahoney N, Mazo R, Barrera AZ, Siemer CP, Gladstone TR, et al. Internet-based depression prevention over the life course: a call for behavioral vaccines. Psychiatr Clin North Am 2011 Mar;34(1):167-183 [FREE Full text] [CrossRef] [Medline]
    70. Hertel PT, Mathews A. Cognitive bias modification: past perspectives, current findings, and future applications. Perspect Psychol Sci 2011 Nov;6(6):521-536. [CrossRef] [Medline]
    71. Conley CS, Durlak JA, Shapiro JB, Kirsch AC, Zahniser E. A meta-analysis of the impact of universal and indicated preventive technology-delivered interventions for higher education students. Prev Sci 2016 Dec;17(6):659-678. [CrossRef] [Medline]
    72. Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin Psychol Psychother 2013;20(1):10-27. [CrossRef] [Medline]
    73. Robichaud M, Dugas MJ, Conway M. Gender differences in worry and associated cognitive-behavioral variables. J Anxiety Disord 2003;17(5):501-516. [Medline]


    ANCOVA: analysis of covariance
    CACE: complier average causal effect
    CBT: cognitive behavioral therapy
    CMD: common mental health disorder
    GAD: generalized anxiety disorder
    GP: general practitioner
    HR: hazard ratio
    i-RFCBT: Web-based rumination-focused cognitive behavioral therapy
    IRR: incidence rate ratio
    ITT: intention-to-treat
    MDE: major depressive episode
    MI: multiple imputation
    PHQ: Patient Health Questionnaire
    PSQ: Psychosis Screening Questionnaire
    PSWQ: Penn State Worry Questionnaire
    RCT: randomized controlled trial
    RESPOND: Reducing Stress and Preventing Depression
    RFCBT: rumination-focused cognitive behavioral therapy
    RNT: repetitive negative thought
    RRS: Ruminative Response Scale of the Response Styles Questionnaire
    RT: repetitive thought
    SCID-I: Structured Clinical Interview for DSM-IV

    Edited by G Eysenbach; submitted 22.06.18; peer-reviewed by M Deady, L Mclellan, J Apolinário-Hagen; comments to author 06.09.18; revised version received 15.03.19; accepted 24.03.19; published 13.05.19

    ©Lorna Cook, Mohammod Mostazir, Edward Watkins. Originally published in the Journal of Medical Internet Research (, 13.05.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.