Published on in Vol 19, No 3 (2017): March

Assessing Feasibility and Acceptability of Web-Based Enhanced Relapse Prevention for Bipolar Disorder (ERPonline): A Randomized Controlled Trial

Assessing Feasibility and Acceptability of Web-Based Enhanced Relapse Prevention for Bipolar Disorder (ERPonline): A Randomized Controlled Trial

Assessing Feasibility and Acceptability of Web-Based Enhanced Relapse Prevention for Bipolar Disorder (ERPonline): A Randomized Controlled Trial

Original Paper

1Spectrum Centre, Faculty of Health and Medicine, Lancaster University, Lancaster, United Kingdom

2Department of Psychology, Northumbria University, Newcastle Upon Tyne, United Kingdom

3Institute of Psychology, Health and Society, University of Liverpool, Liverpool, United Kingdom

4Department of Bio-statistics and Medical Informatics, Acibadem University, Istanbul, Turkey

5CHICAS, Faculty of Health and Medicine, Lancaster University, Lancaster, United Kingdom

6Cumbria Partnership NHS Foundation Trust, Penrith, Cumbria, United Kingdom

7Department of Psychiatry and Applied Psychology, Institute of Mental Health, University of Nottingham, Nottingham, United Kingdom

Corresponding Author:

Fiona Lobban, PhD

Spectrum Centre

Faculty of Health and Medicine

Lancaster University

Bailrigg Campus

Lancaster, LA14YT

United Kingdom

Phone: 44 01524593752

Fax:44 0000000000


Background: Interventions that teach people with bipolar disorder (BD) to recognize and respond to early warning signs (EWS) of relapse are recommended but implementation in clinical practice is poor.

Objectives: The objective of this study was to test the feasibility and acceptability of a randomized controlled trial (RCT) to evaluate a Web-based enhanced relapse prevention intervention (ERPonline) and to report preliminary evidence of effectiveness.

Methods: A single-blind, parallel, primarily online RCT (n=96) over 48 weeks comparing ERPonline plus usual treatment with “waitlist (WL) control” plus usual treatment for people with BD recruited through National Health Services (NHSs), voluntary organizations, and media. Randomization was independent, minimized on number of previous episodes (<8, 8-20, 21+). Primary outcomes were recruitment and retention rates, levels of intervention use, adverse events, and participant feedback. Process and clinical outcomes were assessed by telephone and Web and compared using linear models with intention-to-treat analysis.

Results: A total of 280 people registered interest online, from which 96 met inclusion criteria, consented, and were randomized (49 to WL, 47 to ERPonline) over 17 months, with 80% retention in telephone and online follow-up at all time points, except at week 48 (76%). Acceptability was high for both ERPonline and trial methods. ERPonline cost approximately £19,340 to create, and £2176 per year to host and maintain the site. Qualitative data highlighted the importance of the relationship that the users have with Web-based interventions. Differences between the group means suggested that access to ERPonline was associated with: a more positive model of BD at 24 weeks (10.70, 95% CI 0.90 to 20.5) and 48 weeks (13.1, 95% CI 2.44 to 23.93); increased monitoring of EWS of depression at 48 weeks (−1.39, 95% CI −2.61 to −0.163) and of hypomania at 24 weeks (−1.72, 95% CI −2.98 to −0.47) and 48 weeks (−1.61, 95% CI −2.92 to −0.30), compared with WL. There was no evidence of impact of ERPonline on clinical outcomes or medication adherence, but relapse rates across both arms were low (15%) and the sample remained high functioning throughout. One person died by suicide before randomization and 5 people in ERPonline and 6 in WL reported ideas of suicide or self-harm. None were deemed study related by an independent Trial Steering Committee (TSC).

Conclusions: ERPonline offers a cheap accessible option for people seeking ongoing support following successful treatment. However, given high functioning and low relapse rates in this study, testing clinical effectiveness for this population would require very large sample sizes. Building in human support to use ERPonline should be considered.

Trial registration: International Standard Randomized Controlled Trial Number (ISRCTN): 56908625; (Archived by WebCite at

J Med Internet Res 2017;19(3):e85



Bipolar Disorder

Bipolar disorder (BD) is a lifelong mental health condition characterized by extreme fluctuating mood including recurrent episodes of depression and mania, which generally starts in adolescence and affects approximately 1-1.5% of adults worldwide [1]. The impact of BD on employment and relationships can be devastating, and the condition has high financial costs, estimated at £5.2 billion annually in England alone [2]. Preventing relapse is a key goal of most interventions for BD. Interventions that teach people to recognize and respond to early warning signs (EWS) are recommended by clinical guidelines worldwide [3-5] but implementation in routine clinical practice is poor [6]. Enhanced Relapse Prevention (ERP), a structured manualized intervention for frontline care staff, has shown significant benefit and is well received by patients and staff [7]. However, delivered face-to-face it will only ever be available to a small percentage of people with BD due to low rates of psychological intervention provision even among those who remain in secondary care services. In this study, we test the feasibility and acceptability of a Web-based version of ERP: ERPonline. Web-based interventions in mental health offer the potential to broaden access, reduce waiting times, delivery costs and stigma, and improve quality through standardized delivery [8,9]. There is growing evidence for short-term benefits of Internet-delivered psychological treatments for depression and anxiety disorders compared with waitlist (WL) controls [10], although understanding their implementation into real-world services is still in its infancy [11]. In BD, the evidence, while promising, is at an earlier stage, comprising small-scale feasibility studies [12-18]. These studies, along with results from an international multisite survey [19], suggest that people with BD can use, and are interested in further using, Web-based mental health support. However, detailed evidence is lacking on what kinds of psychosocial support can be Web-based, the best ways to deliver these, who accesses Web-based interventions, what processes and outcomes are impacted on, and how to best design rigorous trials to evaluate them on the Web. This information is essential to inform definitive clinical and cost-effectiveness trials. This study addresses these issues in a novel randomized controlled trial (RCT) to assess feasibility and acceptability of ERPonline with all recruitment and assessments of outcome performed remotely.


The objectives of this study were as follows:

First, to assess the feasibility of (1) creating a Web-based version of enhanced relapse prevention for BD (ERPonline) and (2) an RCT design using Web-based and telephone data collection to evaluate effectiveness.

Second, to determine the acceptability of ERPonline for people with BD via (1) ERPonline website usage, (2) number and type of adverse events associated with site use, and (3) detailed feedback from participants about their experiences of ERPonline to inform future developments.

Third, to determine the feasibility and acceptability of data collection via the Internet and telephone measured by recruitment and retention rates, data completion, and direct feedback from participants.

Fourth, to test the impact of the intervention on hypothesized mechanisms of change to understand processes underlying any impact.

Finally, to estimate the likely effect size of the intervention on a range of outcomes, particularly noting any negative impacts.


A single-blind RCT with nested qualitative study comparing ERPonline plus usual treatment with a “waitlist control” arm with delayed access to ERPonline plus usual treatment. Primary outcomes were feasibility and acceptability. Process and clinical outcomes were assessed to identify measures sensitive to change collected remotely and to explore potential positive and negative impacts of the intervention. Remote data collection and online recruitment increased the external validity of the trial by encouraging participation from those unable or unwilling to engage in face-to-face clinical trials, who are also more likely to be those people unable or unwilling to engage in face-to-face clinical support for whatever reason. The study was not powered to test statistically significant impact. The trial was preregistered and full protocol published [16]. Ethics approval was given by UK National Research Ethics Service (NRES) Committee North West (Ref 12/NW/0594).


We aimed to recruit 125 participants, anticipating a dropout rate of up to 35% (based on retention rates from previous trials of Web-based interventions for BD [12-18]) providing 40 people per arm, sufficient to meet the aims of our study to assess feasibility and acceptability. Participants were aged ≥18 years, residents of the United Kingdom, with a confirmed diagnosis of BD (1 or 2), at risk of relapse (≥3 previous episodes, ≥1 in the preceding 2 years), and with access to the Internet. We excluded people in current episode (within previous 4 weeks), currently under Mental Health Act section and therefore likely to be in current episode or at high risk of harm to self or others, or unable to understand English sufficiently to engage with the study.

Recruitment Strategy

The study was presented to clinical teams in 8 NHS Mental Health Trusts in England, and staff were reminded in monthly team meetings to direct service users to the Web-based registration site. An advert was placed in a UK charity newsletter (Bipolar UK), and on a charity website (Bipolar Scotland). A link to ERPonline was put in NHS Choices, and British Broadcasting Corporation (BBC) health online presented a short article that linked to the website. The research team regularly tweeted about the study, and our service user lead was interviewed on local radio about the study.

People were invited to visit the site which explained the study, allowed them to check eligibility, and to register an interest in participating. Participants read online participant information sheet and completed an online consent form. Consent and capacity were reassessed at each assessment point.


ERPonline was developed with extensive input from a reference group of 8 adults with BD to adapt the original ERP manual to a Web-based format. Input (online and face-to-face) occurred throughout the study, but was more extensive during the initial development of the ERPonline site and included feedback on content of draft modules, user testing of the ERPonline website, and providing video and case material of lived experience which are integral parts of the intervention site. The aim of the intervention is to help people develop a coherent working model of their mood changes, recognize and manage triggers and EWS, and develop coping strategies to manage these effectively. Key modules are summarized in Table 1 with more detail in the protocol paper [20].

Table 1. Key intervention modules in Web-based enhanced relapse prevention intervention (ERPonline).
SectionModule titleModule descriptionERPonline (n=47) average
number of module views
per person
ERPonline (n=47) average
time spent per module
per person (min)
Mean (SDa)
Medianb (min-max)
Mean (SDa)
Medianb (min-max)
How to use the
Ways to navigate the site to
get the best from the
available modules
7.43 (5.77)
7 (0-25)
8.61 (8.70)
7.5 (0-46.5)

IntroductionExplains what ERPonline is,
rationale for this approach,
why it might be useful, and
how to involve a relative or
friend if desired
4.28 (5.46)
3 (0-30)
5.88 (8.99)
2 (0-40)

What is bipolar?Background information about
what bipolar disorder is, theories about causes, common
consequences, and an overview
of available treatments

8.00 (7.34)
7 (0-30)
11.95 (13.51)
7 (0-52.5)
Mood chartingHow to use a Web-based tool to
monitor mood on a daily basis to
help recognize normal mood
fluctuation and pick up early signs
of a mood episode
13 (0-2519)
122.89 (386.43)
14.5 (0-2150.5)

Life chartingComplete a chart of past mood
episodes, identifying potential
triggers and coping strategies for
future mood changes
49.09 (147.55)
11 (0-990)
47.34 (114.45)
8.00 (0-744)

Identifying triggersDetailed analysis of triggers of
previous mood episodes, followed
by a personalized plan of how to
manage triggers
11.34 (25.57)
1 (0-148)
15.07 (30.09)
0.5 (0-140.5)
Early warning signs (EWS)
-high mood
Detailed analysis of EWS of high
mood to develop a relapse
signature for (hypo) mania
14.47 (26.72)
0 (0-90)
17.36 (32.31)
0 (0-114)

Coping strategies
-high mood
Review of current strategies to
manage high mood and
introduction to new strategies that
may be helpful
5.68 (10.62)
0 (0-44)
10.0 (19.78)
0 (0-71.5)

Early warning signs (EWS)
-low mood
Detailed analysis of EWS of low
mood to develop a relapse
signature for depression
9.26 (22.47)
0 (0-94)
9.38 (21.87)
0 (0-84.5)

Coping strategies
-low mood
Review of current strategies to
manage low mood and
introduction to new strategies that
may be helpful
3.83 (11.26)
0 (0-59)
3.95 (11.23)
0 (0-52.5)
things up
Staying well
Identifying and managing stress
Understanding the importance of
social rhythms and how to
regulate these to manage mood
How relationships with other
people impact on mood
4.06 (6.11)
0 (0-25)
4.39 (8.30)
0 (0-37)

Your staying well
An individualized summary of
staying well strategies, early
warning signs to look out for, and
coping strategies to regulate mood
3.09 (5.94)
0 (0-30)
2.35 (4.60)
0 (0-18.5)

aSD: standard deviation.

bThe median value 0 indicates that at least half the sample did not visit this module.

Each module included information, suggested strategies, and case examples. Users interacted with the site to input personal information relevant to their own triggers, EWS, and coping strategies. These informed an individualized staying well plan. The site also provided signposting to additional formal and informal support. Participants were free to choose the order they visited modules (although they were listed in logical order), and were invited to involve a supporter of their choosing. Each module included recommendations of how the supporter could be involved in relapse prevention. All participants continued to receive any other treatment as usual throughout the study. The home page is shown in Figures 1 and 2 for illustration.

Figure 1. The online enhanced relapse prevention intervention (ERPonline) home page - top of page.
View this figure
Figure 2. The online enhanced relapse prevention intervention (ERPonline) home page - bottom of the page.
View this figure

Procedure: Randomization and Masking

Diagnostic eligibility was confirmed using Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) [21] administered by telephone by trained research assistants. Training consisted of scoring training videos, conducting clinical sensitive interviews on the telephone, and recording an interview with someone with BD who provided experiential feedback and which was then rated by a supervising clinical academic. Training continued until ratings were reliable and clinical style was of high quality. Monthly supervision to ensure reliability in scoring of telephone interviews continued throughout the study. Following baseline assessments (telephone and Web-based) participants were randomly allocated by an independent clinical trials unit (CTU) using 1:1 ratio, minimized on number of previous episodes (<8, 8-20, 21+) and including a random element to minimize predictability of allocation. Those in the ERPonline arm received an email containing a weblink and instructions of how to log-on to the site using a unique username and password. Control participants received an email or telephone call informing them of the allocation and emphasizing the importance of continued participation throughout the trial. All communication with CTU and participants regarding randomization was conducted by the trial manager (unblinded). All communication with participants reiterated the importance of not telling the researcher carrying out the follow-up interviews which group they were in, and why this was important. All assessments were conducted by blind researchers. Blindness was further maintained using restricted file access to any data showing randomization, and prefacing each follow-up interview with a reminder about why it was important not to say anything about which arm they were in.

During the trial, participants were sent an additional email inviting them to provide qualitative feedback. The ERPonline group was asked to complete a Web-based survey about their views of the ERPonline site and any improvements they would recommend. They were also given the option (between 2 and 12 months following randomization) to take part in a telephone interview about their experiences of using ERPonline. Given the relatively novel primarily Web-based trial design, the WL control group was sent a survey before accessing the site about their reasons for engaging in the trial, and their experience of taking part in the trial.

A reflective log detailed our experiences throughout the trial.


Proposed mechanisms of change, were assessed at baseline, 24 and 48 weeks including frequency of EWS monitoring (EWS checklist for relapse in depression and mania [22] Likert scale 1=never to 4=very regularly), adapted Brief Illness Perception Questionnaire (BIPQ score 0-110: higher score=more negative beliefs) [23], and the Medication Adherence Rating Scale (MARS 0-10: higher score=higher compliance) [24]. These measures were either designed for use with people with BD (EWS checklists and BIPQ), or have been successfully used with this population (MARS). They are all self-report, have high face validity, and have been shown to be valid and reliable measures, making them highly applicable to Web-based use.

Interviewer-rated outcome measures were administered by telephone by two trained research assistants, at baseline, 12-, 24-, 36-, and 48-week follow-up. These included SCID-LIFE [25] providing a retrospective weekly rating of depression (1-6) and mania (1-6) (scores of 5 or 6 indicate major mood episode); Hamilton Depression Rating Scale (HAM-D scores above 7 indicate mild depression, above 13 moderate, and above 18 severe) [26], and Mania Rating Scale (MRS scores of 11 and above indicate hypomania) [27]; the Personal and Social Performance Scale (PSP scores 70-80 indicate mild difficulties, above 80 is good functioning) [28]; and the Multidimensional Scale of Independent Functioning (MSIF Likert scale 1=normal functioning, to 7=total disability) [29]. Self-report outcome measures were collected at baseline, 24, and 48 weeks and included the Work and Social Adjustment Scale (WSAS less than 10=subclinical; 10-20 some functional impairment; above 20 moderate psychopathology) [30]; Quality of Life in Bipolar Disorder (QoLBD range 48-240 with high score=higher quality of life) [31]; and the Bipolar Recovery Questionnaire (BRQ score 0-3600, high score=higher recovery) [32]. Web-based versions of the EQ5D5L [33] and the Client Service Receipt Inventory (CSRI) [34] were piloted to assess the feasibility of collecting this data on the Web and to test the sensitivity to change in this population as neither have been previously used in this format. A checklist to record current treatment was developed for the study to define usual treatment.

The only change to the published protocol was to record only the frequency of monitoring of EWS for hypomania and depression, as early feedback from participants indicated the full checklist was too long. All serious adverse events (SAEs) were recorded and reported to the Trial Steering Committee (TSC). All participants were given a £10 shopping voucher on completion of measures at each assessment point.


Descriptive statistics report the characteristics of the sample recruited; use of the website; and rates of recruitment, retention, and data completion in each arm of the trial. The impact on repeated process and outcome measures was tested using linear models with correlated errors, which allow for correlation between repeated measures from the same participant. For ordinal data, we used generalized linear mixed models. We report both unadjusted analyses, and those adjusting for any differences in baseline demographic (age, gender, ethnicity, employment, education) and clinical variables (number of previous episodes, and whether or not prescribed a mood stabilizer).

Incomplete records from participants were retained, and analyses used maximum likelihood estimation for all model parameters. Statistical comparison of outcomes was made between the two trial arms at 24 and 48 weeks follow-up.

Weekly ratings of depression and mania from the SCID-LIFE were used to analyze time to first relapse (any and separately for depression [requiring 2 consecutive weeks], or mania [1 week]) and the proportion of time spent in episode (defined as SCID-LIFE rating of 5 or 6), or in subsyndromal state (SCID-LIFE rating of 3 or 4) or euthymic (SCID-LIFE rating of 1 or 2). To analyze the impact of the intervention on time to first relapse, we used a Cox’s proportional hazards regression model. Beta regression was used to compare the proportion of time spent in episode or subsyndromal or euthymic in each arm.

The study is not powered to test for statistically significant impact and therefore we do not specify a primary outcome, or set a level of statistical significance for interpreting analyses. All analyses were run from R open-source computing environment version 3.3.1 (R Foundation for Statistical Computing).

Content analysis of qualitative survey data highlighted the individual points made and these were grouped into key themes. Interview transcripts were analyzed in depth using indexing and charting methods inspired by Framework Analysis [35]. All transcripts were independently coded by the interviewer (MG) and a second member of the research team. Codes were compiled into a tentative coding frame with thematic headings. Narrative summaries were created from each of the conceptual themes across all cases. This data will be reported in full elsewhere but here we present key data relevant to the feasibility and acceptability aims of the trial.

Quality Assurance

Only two unblindings occurred. In both instances, the participant inadvertently indicated their group during a telephone assessment (one WL control, one ERPonline). Subsequent assessments were completed by a blind Research assistant. At each follow-up, 10% of the SCID-LIFE interviews were rated by both researchers and kappa statistic calculated to assess interrater reliability. These ranged from acceptable (κ=.54, 95% CI 0.39-0.69, at 36 weeks based on 142 weekly ratings for 6 participants) to high (κ=.90, 95% CI 0.82-0.98, at 12 weeks based on 192 weekly ratings for 8 people).

Feasibility and Acceptability of Trial Design

Participant flow is detailed in Figure 3, including recruitment, over half of which came via online sources. A total of 96 people were randomized (49 to WL, 47 to ERPonline) over a 17-month period, with 80% retention in telephone and online follow-up at all time points, except week 48 (76%). Attrition was 11% lower in WL arm.

Participants were predominantly diagnosed with BD1 (90%) and had a chronic relapsing course (67% had over 21 relapses; see Table 2). Despite this, the group was currently high functioning and had a positive attitude to recovery. The vast majority were taking and adherent to medication to manage their mood and over half had previously received psychological treatment for BD (where specified, this was most commonly described as cognitive behavior therapy [CBT]).

Table 2. Key characteristics of participant sample at baseline.
Participant characteristicsWait list (n=49)ERPonline (n=47)
Baseline demographic and clinical variables

Age, mean (SDa)43.8 (11.45)42 (12.23)
Gender, n (%)

Female32 (65)27 (57)
Ethnicity, n (%)

White British44 (90)38 (81)

Any other white2 (4)5 (11)

Black British-1 (2)

Caribbean1 (2)-

Asian British1 (2)-

Indian-1 (2)

Any other mixed-1 (2)

Missing1 (2)1 (2)
Occupational status, n (%)

Full-time paid or self16 (33)21 (45)

Part-time paid or self13 (27)6 (13)

Voluntary3 (6)4 (9)

Not employed10 (20)6 (13)

Student3 (6)2 (4)

Housewife or househusband-3 (6)

Retired4 (8)5 (11)
Education, n (%)

No formal qualifications1 (2)-

CSEb or O Level or GCSEc5 (10)4 (9)

A Level7 (14)7 (15)

Degree17 (35)16 (34)

PGd Diploma or qualification13 (27)13 (28)
Doctorate or PhD3 (6)7 (15)
Total number of past episodes (baseline), n (%)

<76 (12)2 (4.)

8-2012 (25)12 (26)

21+31 (63)33 (70)
Taking mood stabilizer (baseline), n (%)

Not on any medication (so item rated not applicable)3 (6)2 (4)

No mood stabilizer15 (31)11 (23)

Lithium12 (25)11 (23)

Sodium valproate5 (10)14 (30)

Carbamazepine3 (6)1 (2)

Lamotrigine11 (22)8 (17)
Treatment history (baseline), n (%)

Ever used mental health services46 (94)42 (89)

Clinical diagnosis of BD1e (vs BD2)44 (90)44 (94)

Ever seen a psychiatrist46 (94)43 (92)

Ever prescribed medication for BD49 (100)46 (98)

Currently taking medication for bipolar disorder40 (82)40 (85)

Ever received therapy or psychosocial intervention for bipolar disorder26 (53)27 (57)

Currently receiving therapy for bipolar disorder8 (16)10 (21)
Process measures

Early warning signs monitoring frequency—depression, n (%)

Never3 (6)3 (6)

Occasionally18 (37)11 (23)

Fairly regularly8 (37)20 (43)

Very regularly10 (20)13 (28)
Early warning signs monitoring frequency—hypomania, n (%)

Never6 (12)5 (11)

Occasionally21 (43)14 (30)

Fairly regularly16 (33)18 (38)

Very regularly6 (12)10 (21)
Brief Illness Perception Questionnaire—total (high score=more negative model), mean (SD)60.6 (10.5)60.7 (9.9)
Medication Adherence Rating Scale, mean (SD)6.9 (2.2)7.0 (2.1)
Outcome measures

Hamilton Depression Rating Scale, mean (SD)4.5 (5.3)3.5 (4.2)
Mania Rating Scale, mean (SD)1.3 (2.4)1.0 (1.7)
Personal and Social Performance Scale, mean (SD)79.7 (11.2)79.1 (13.1)
Multidimensional Scale of Independent Functioning, n (%)25 (51)
14 (29)
8 (16)
2 (4)
0 (0)
0 (0)
21 (45)
16 (34)
6 (13)
2 (4)
2 (4)
0 (0)
Work and Social Adjustment Scale, mean (SD)11.4 (9.4)12.8 (8.7)
Quality of Life in Bipolar Disorder, mean (SD)162.7 (33.5)162.5 (22.8)
Bipolar Recovery Questionnaire, mean (SD)2332 (394)2342 (383)

aSD: standard deviation.

bCSE: Certificate of Secondary Education.

cGCSE: General Certificate of Secondary Education.

dPG: postgraduate.

eBD: bipolar disorder.

Figure 3. The online enhanced relapse prevention intervention (ERPonline) consort diagram.
View this figure

Feasibility and Acceptability of Data Collection

Telephone and Web-based data collection procedures were generally acceptable to participants based on information from 41 participants (WL survey, n=22; ERPonline interviews, n=19 reported separately). Survey data indicated factors that encouraged people to take part including opportunity to improve their own resilience and self-management, wanting to help others, and recognizing the importance of research on improving Web-based interventions, due to a perceived gap in face-to-face services and some existing websites feeling unsafe. Many of these factors were also cited as facilitating retention in the trial, as well as factors such as text reminders about follow-ups, and viewing the research team as sensitive, polite, and nonintrusive. Participants reported that the research process was well-managed, clearly explained, straightforward, and flexible, and they liked the shopping vouchers. Some participants believed completing the measures had changed their thinking about their mood and diagnosis. Barriers to retention included procedural difficulties, such as remembering follow-up times and rescheduling missed appointments, feeling “weird” to have interviews only on the phone and difficulties finding private space for the phone calls; issues with measures, some of which were too long (CSRI) and could be tiring, distressing, and required recall over long periods of time; and technical difficulties with Web-based questionnaires. Some reported feeling disappointed to be in the WL control arm, although had remained in the trial.

Additional key data collection lessons we learnt included the importance of checking electronic communication is received (some of our reminder emails were initially going into junk folders), the need to accommodate the high demand for evening and weekend telephone appointments, and the importance of text reminders for telephone appointments.

Feasibility and Acceptability of ERPonline Intervention

ERPonline is low cost at an estimated £19,340 to create, and approximately £2176 per year to host and maintain the site. Development costs included time to adapt content from the ERP manual; discussion and feedback with coauthors; Web developer time to build the site; filming and producing videos; and costs for the Service User Reference Group to feedback on early iterations. Hosting costs include software updates and technical issues (estimated at 2 h per week) and space on a server. To keep costs low, ERPonline was delivered unsupported and without a Web-based moderated forum, despite these being part of the intended design.

Activity levels were highly skewed. Two people allocated to ERPonline never visited the site. Mean number of page views per person was 259 (SD 577), median was 85 (range 0-3203). Participants spent a mean of 259 min (SD 509), median 76 min (range 0-2770) accessing ERPonline throughout the 48-week intervention period. The most frequently viewed modules were “Life Charting” (median views 11; range 0-990 per person) and “Mood Charting” (median views 13; range 1-2519 per person), which is unsurprising as they offered an ongoing monitoring function. “Coping strategies for Low Mood” (median views 0: range 0-44 per person) and “Your Staying Well Plan” (median views 0; range 0-30 per person) were the least frequently visited, but also occurred toward the end of the listed modules (see Table 1 for number of visits and time spent on all modules).

A total of 17 ERPonline participants (36%) responded to the Web-based survey. Overall these participants were satisfied (13/17 people [76%] somewhat or very satisfied), found it somewhat or very helpful (12/17, 71%), and very or extremely relevant (13/17, 76%). Only one person said they would not recommend it to a friend. Most useful features were recognizing EWS of relapse, shared experiences through videos, mood monitoring, ability to revisit and refresh skills, improved knowledge and self-management of BD, ease of use, and being able to use the site with the family. The key recommendation for improvement was additional support with working through the materials. The sample of questionnaire respondents described themselves as confident (n=16, 94% very or extremely confident) and regular (n=12, 71% at least daily) Internet users.

A total of 19 people took part in qualitative interviews about their experience of the trial and use of the ERPonline site. The key finding was the importance of relationships that the individual developed with the ERPonline team in determining retention into the study and use of the site:

I think the sort of general thoroughness and kindness of the people I dealt with that certainly contributed to, you know, me sort of staying in the study. Everybody’s been really upbeat, very positive, very accommodating

This is particularly interesting when we consider there was no face-to-face contact. Sole direct contact was by telephone at three monthly interviews for SCID-LIFE interviews, which for some was preferable to face-to-face:

…I think it being over the ‘phone makes it a bit easier. If it’s face-to-face I would have probably not been quite so comfortable answering. But yeah over the ‘phone was definitely not so bad.

Crucial to the strength of the alliance was the perceived trustworthiness of the team and being made aware of the extensive user involvement in design and content of the site:

It’s always available and also the information’s on there has been put together by the people who do know what they’re doing.
I suppose the prospect of the online study kept me quite interested and the fact that it was developed by other people with Bipolar and that was something that I was definitely interested in…

A key recommendation to improve the ERPonline site was to integrate human support to facilitate ongoing use of the site. This is consistent with previous studies in which adherence was higher in groups receiving a Web intervention plus support as compared with Web intervention only [31]. Participants felt websites should be used to support interventions delivered by real people and not as a cheaper replacement:

The cash strapped health service will rely heavily on these sort of techniques which I think only fill one part of the market. I think they only really deal with, you know, and a comparatively narrow field of potential patients. I think they’re very useful but I do think the gold standard involves some sort of face-to-face psychological therapy. And I think the clinical literature bears that out so, I want more jobs for psychologists basically
[Henry: 23.893-23.898]
Table 3. Descriptive statistics on outcome and process measures at 24- and 48-week follow-ups.
VariableGroupBaseline, n
WLa=49(Ic); 49(Od)
ERPb=47(I); 47(O)

12 weeks
ERP=43 (I)

24 weeks
WL=42(I); 43(O)
ERP=39(I); 34(O)

36 weeks
48 weeks
WL=45(I); 42(O)
ERP=36(I); 31(O)

Process measures
Early warning signs monitoring
frequency—depression (O),
n (%)
never WL3 (6)
8 (19)
4 (10)
Occasionally18 (37)
13 (30)
11 (26)
Fairly regularly18 (37)
17 (40)
19 (45)
Very regularly10 (20)
5 (12)
6 (14)

never ERP3 (6)
1 (3)
1 (3)
Occasionally11 (23)
11 (32)
6 (19)
Fairly regularly20 (43)
15 (44)
10 (32)
Very regularly13 (28)
7 (21)
14 (45)
Early warning signs monitoring
frequency—hypomania (O),
n (%)
never WL6 (12)
11 (26)
4 (10)
Occasionally21 (43)
17 (40)
18 (43)
Fairly regularly16 (33)
9 (21)
11 (26)
Very regularly6 (12)
6 (14)
7 (17)

never ERP5 (11)
2 (6)
1 (3)
Occasionally14 (30)
10 (29)
8 (26)
Fairly regularly18 (38)
15 (44)
11 (35)
Very regularly10 (21)
7 (21)
11 (35)
BIPQe—total (O),
mean (SDf)
WL60.6 (10.5)
49.4 (22.6)
49.4 (24.6)
ERP60.7 (9.9)
39.3 (27.0)
36.2 (29.1)
MARSg (O),
mean (SD)
WL6.9 (2.2)
6.7 (2.6)
6.6 (2.5)
ERP7.0 (2.1)
6.8 (2.7)
7.0 (2.2)
Outcome measures
HAM-Dh (I),
mean (SD)
WL4.5 (5.3)7.5 (7.4)7.3 (8.6)6.8 (8.6)8.2 (9.0)
ERP3.5 (4.2)6.6 (6.7)6.9 (8.0)6.0 (8.3)7.1 (9.3)
MRSi (I),
mean (SD)
WL1.3 (2.4)2.7 (3.7)2.2 (4.1)1.7 (2.9)1.7 (2.2)
ERP1.0 (1.7)2.5 (4.4)2.4 (3.9)2.0 (4.0)1.4 (2.4)
PSPj (I),
mean (SD)
WL79.7 (11.2)75.0 (15.1)76 (16.4)79.8 (14.8)78.4 (15.6)
ERP79.1 (13.1)77.8 (15.1)76.7 (15.4)77.8 (16.1)80.7 (16.1)
MSIFk-global (frequencies for scores categories 1, 2, 3, 4, 5, 6) (I),
n (%)

125 (51)24 (59)20 (48)25 (61)28 (62)
214 (29)8 (20)13 (31)10 (24)9 (20)
38 (16)4(10)5 (12)3 (7)6 (13)
42 (4)5 (12)3 (7)1 (2)1 (2)
50 (0)0 (0)1 (2)2 (5)0 (0)
60 (0)0 (0)0 (0)0 (0)1 (2)

121 (45)26 (60)18 (46)20 (56)22 (61)
216 (34)10 (23)12 (31)9 (25)9 (25)
36 (13)3 (7)6 (15)5 (14)4 (11)
42 (4)3 (7)2 (5)0 (0)1 (3)
52 (4)1 (2)1 (3)2 (6)0 (0)
60 (0)0 (0)0 (0)0 (0)0 (0)
WSASl (I),
mean (SD)
WL11.4 (9.4)
12.4 (10)
12.9 (10.6)
ERP12.8 (8.7)
14.3 (9.1)
14.8 (10.3)
QoLBDm (O),
mean (SD)
WL162.7 (33.5)
161.2 (39.7)
154.9 (36.1)
ERP162.5 (22.8)
156.5 (33.4)
151.8 (41.7)
BRQn (O)
mean (SD)
WL2332 (394)
2309 (504)
2336 (468)
ERP2342 (383)
2451 (430)
2414 (577)

aWL: waitlist.

bERP: enhanced relapse prevention.

cI: interviewer rated by telephone.

dO: completed online.

eBIPQ: Brief Illness Perception Questionnaire (score 0-110 higher score=more negative beliefs).

fSD: standard deviation.

gMARS: Medication Adherence Rating Scale (0-10 higher score=higher compliance).

hHAM-D: Hamilton Depression Rating Scale (scores above 7 indicate mild depression, above 13 moderate, and above 18 severe).

iMRS: Mania Rating Scale (scores of 11 and above indicate hypomania).

jPSP: Personal and Social Performance Scale (scores 70-80 indicate mild difficulties, above 80 is good functioning).

kMSIF: Multidimensional Scale of Independent Functioning (Likert scale 1=normal functioning, to 7=total disability).

lWSAS: Work and Social Adjustment Scale (less than 10=subclinical; 10-20 some functional impairment; above 20 moderate psychopathology).

mQoLBD: Quality of Life in Bipolar Disorder (range 48-240 with high score=higher quality of life).

nBRQ: Bipolar Recovery Questionnaire (score 0-3600 high score=higher recovery).

Table 4. Comparison of linear models with correlated errors to test for differences between waitlist (WL) and Web-based enhanced relapse prevention intervention (ERPonline) on outcome and process measures at 12-, 24-, and 48-week follow-ups. Unadjusted model showing estimates of difference between beta estimates at each time point.
VariableModel 1—Unadjusted analysis

12-week follow-up
95% CI
(P value)
24-week follow-up
95% CI
(P value)
36-week follow-up
95% CI
(P value)
48-week follow-up
95% CI
(P value)
Early warning signs monitoring

0.73−1.86 to 0.40 (.20)

−1.39−2.61 to −.163 (.03)
Early warning signs monitoring

−1.72−2.98 to −.47 (.01)

−1.61−2.92 to −.30 (.02)
Brief Illness Perception Questionnairea —total

10.700.90 to 20.5 (.03)

13.182.44 to 23.93 (.02)
Medication Adherence Rating Scale

−0.102−1.07 to 0.87 (.84)

−0.327−1.34 to 0.685 (.53)
Personal and Social Performance Scale−2.91−9.19 to 3.37 (.36)−0.198−6.77 to 6.38 (.95)0.77−5.87 to 7.41 (.82)−2.87−9.27 to 3.52 (.38)
Multidimensional Scale of Independent Functioning—global (frequencies for scores categories 1, 2, 3, 4, 5, 6)0.169−0.867 to 1.20 (.75).029−0.959 to 1.02 (.95)−0.074−1.15 to 1.00 (.89).281−0.785 to 1.35 (.61)
Work and Social Adjustment Scale

−1.61−5.67 to 1.46 (.30)

−0.53−3.90 to 2.83 (.76)
Quality of Life in Bipolar Disorder

6.67−5.73 to 19.1 (.29)

3.99−9.72 to 17.7 (.57)
Bipolar Recovery Questionnaire

−52.3−195 to 90.7 (.47)

−38.09−208.58 to 132.41 (.66)

aBrief Illness Perception Questionnaire total measures how negative a model the person has—so high score=more negative model.

Table 5. Comparison of linear models with correlated errors to test for differences between waitlist (WL) and Web-based enhanced relapse prevention intervention (ERPonline) on outcome and process measures at 12-, 24-, and 48-week follow-ups. Adjusted for any differences in baseline demographic (age, gender, ethnicity, employment, education) and clinical variables (number of previous episodes, and whether or not prescribed a mood stabilizer).
VariableModel 2—Adjusted analysis

12-week follow-up
95% CI
(P value)
24-week follow-up
95% CI
(P value)
36-week follow-up
95% CI
(P value)
48-week follow-up
95% CI
(P value)
Early warning signs monitoring frequency —depression

−0.721−1.85 to 0.413 (.21)

−1.38−2.61 to −0.153 (.03)
Early warning signs monitoring frequency —hypomania

−1.70−2.96 to −0.452 (.01)

−1.60−2.91 to −0.291 (.02)
Brief Illness Perception Questionnairea —total

11.061.25 to 20.9 (.03)

13.62.69 to 24.6 (.02)
Medication Adherence Rating Scale

−0.128−1.11 to 0.855 (.80)

−0 .356−1.39 to 0.674 (.50)
Personal and Social Performance Scale−2.99−8.84 to 2.863 (.32)−0.430−7.02 to 6.16 (.90).704−6.06 to 7.47 (.84)−3.21−9.69 to 3.26 (.33)
Multidimensional Scale of Independent Functioning—global (frequencies for scores categories 1, 2, 3, 4, 5, 6)0.092−0.907 to 1.09 (.86)−0.075−1.03 to 0.878 (.88)−0.084−1.12 to 0.95 (.87)0.275−0.743 to 1.29 (.60)
Work and Social Adjustment Scale

−1.46−4.53 to 1.64 (.36)

−0.40−3.84 to 3.03 (.82)
Quality of Life in Bipolar Disorder

6.23−6.13 to 18.6 (.32)

3.58−10.4 to 17.5 (.62)
Bipolar Recovery Questionnaire

−63.29−206.47 to 79.88 (.39)

−35.84−209.78 to 138.10 (.69)

aBrief Illness Perception Questionnaire total measures how negative a model the person has—so high score=more negative model.

Estimate of Impact on Outcome and Process Measures

Descriptive statistics on process and outcome measures at each time point are shown in Table 3. Comparison between WL and ERPonline on process and outcome measures at 12-, 24-, and 48-week follow-ups are shown in Table 4. Models adjusting for baseline demographic and clinical variables are also shown in Table 5.

Process Measures

ERPonline increased the frequency of monitoring early signs of mood change (EWS—depression and EWS—hypomania), evident for hypomania at 24 weeks (−1.72, 95% CI −2.98 to −0.47), and for both at 48 weeks (depression −1.39, 95% CI −2.61 to −0.163; hypomania −1.61, 95% CI −2.92 to −0.30), and improved working model of mood changes (BIPQ—high score indicates more negative model) at both 24 weeks (10.70, 95% CI 0.90-20.5) and 48 weeks follow-ups (13.18, 95% CI 2.44-23.93; Table 4). All differences are robust to adjustments in model 2 for baseline differences between the groups (Table 5). Medication adherence was high (indicated by high score on the MARS) throughout the study and did not differ between groups.

Mood and Functioning

Depression and hypomania were low at all time points suggesting a generally stable and euthymic group. Similarly, functioning on WSAS, PSP, and MSIF at baselines were suggestive of very mild impairment in work and social performance and remained so throughout. Time spent in euthymic, subsyndromal, and relapse mood states respectively in WL were 93% (SD 8%), 5% (SD 7%), and 3% (SD 15%), and in ERPonline: 95% (SD 8%), 4% (SD 6%), and 2% (SD 4%). There were no notable differences between the two groups in any of the outcome measures at any of the time-points.


Of the 96 participants, one provided no SCID-LIFE data at follow-up. Only 15 (16%) participants experienced a relapse over the 48-week follow-up; 11 (11%) depressive and 7 (7%) mania-type. There were no notable differences between groups on time to any relapse (unadjusted hazard ratio [HR] 1.67, 95% CI 0.60-4.71), P=.33; time to depressive episode (HR 1.53, 95% CI 0.49-4.83), P=.47; time to mania type episode (HR 2.87, 95% CI 0.56-14.8), P=.12. Given the low level relapses, no Kaplan-Meier curves are presented.

Adverse Events

During the trial, 1 participant completed suicide before randomization, 11 participants (11% of those randomized) reported suicidality and self-harm, and 1 made a suicide attempt (before withdrawing from the study). It was found that 6 were in the WL arm, and 5 were receiving ERPonline. None of the SAEs were deemed study related by an independent TSC.

Principal Findings

ERPonline is a novel Web intervention for improving relapse prevention and providing National Institute for Health and Care Excellence (NICE) congruent information to people with BD. This study indicates that the development and evaluation of this type of approach in a rigorous RCT using telephone and Web-based assessments is both feasible and acceptable. Important lessons were learnt relevant to each of our study aims, but which also have relevance to the wider development and evaluation of remote-access approaches for other health problems.

With the help of our Service User Reference Group, we were able to develop a Web-based version of an existing ERP intervention for people with BD at a very low cost that received largely positive feedback, and led to no evident adverse events. Activity was highly skewed but over 90% of our sample visited the site more than once, which can be compared with MyRecoveryPlan [17] that reported site returns for 71% in a coached group, and only 44% in an unsupported group. On the basis of levels of use of the different modules and direct participant feedback, engagement could be enhanced by making the intervention more interactive and providing support to use it.

Recruitment, retention, and data completion strategies were largely successful. Retention was higher than demonstrated in previous Web-based trials with people who are affected with BD [14,15,17]. Key features of the trial design that facilitated this included payment for completing assessments, text and email reminders, a WL control design, and a friendly flexible research team who were willing to offer telephone appointments at times to suit participants including out-of-office hours. However, to reach the sample size required for large scale clinical and cost effectiveness trials, paying for advertising through popular websites such as Google and Facebook may be necessary [36].

Feedback about the experience of taking part in a primarily Web-based trial was mixed. Some participants reported difficulties finding a private space to take telephone calls or finding Web-based measures difficult, tiring or distressing, whereas others valued the flexibility, convenience, and felt more able to be open about the problems they had experienced than in a face-to-face interview. This suggests that trials which offer a choice of data collection options may be most effective in achieving recruitment and retention targets.

However, further work is needed to test the validity and reliability of these data collection approaches. Our data showed that while the hypothesized increase in EWS monitoring and development of more positive beliefs about mood swings did occur in those receiving ERPonline compared with WL control group, we did not see any benefit of ERPonline on any of the clinical outcome measures. This was largely due to the ceiling effect on our outcome measures. Only 16% of the total sample experienced any relapse, compared with expected levels of 50-70% [37].

This ceiling effect was consistent across all outcome measures and all assessors. Therefore, the most likely explanations are either that the method of data collection is leading to underreporting of problems, or that the participating sample reflect a different population from those taking part in more traditionally designed face-to-face clinical studies.

With regards to the first possibility, whereas we did not directly test the reliability of the data compared with a face-to-face interview, other studies have done this comparing telephone and face-to-face interview data of SCID assessments found high levels of agreement [38]. Our team have also carried out a parallel Web-based RCT which included the same Web-based and telephone assessments, delivered through researchers trained by the same methods, and which will report relapse rates of 47% which are akin to those expected from previous research data and much higher than in this study [39].

The second possible explanation can be explored by examining the characteristics of our participants. Compared with bipolar samples recruited to other face-to-face trials [37] including one evaluating clinician delivered ERP [7], and samples in other Web-based trials which all show higher relapse rates [14,15,17], our sample are more euthymic, highly educated, likely to be in employment, and have had surprisingly high levels of access to previous psychological therapy. Further work is needed to better understand how using a primarily Web-based trial design may impact on sample characteristics, and the information they provide.

The Future for ERPonline

ERPonline offers a cheap and easily accessible option for people who are seeking ongoing support following successful treatment, which is currently unavailable. However, given the high functioning and low relapse rates evident in this study, testing the clinical effectiveness of ERPonline for this population would require very large sample sizes. Alternatively, ERPonline could target people at an earlier stage of treatment, who have had not yet received more expensive face-to-face psychological therapy, and need support to understand their mood swings, consider the pros and cons of medication use, and explore the usefulness of monitoring and managing EWS of relapse. For this group, ERPonline may offer a way to reduce the need for expensive individual therapy. Consistent with participant recommendations and previous research, we also need to consider how best to integrate support mechanisms to facilitate use of the intervention, either by integrating the Web-based resource with clinician delivered relapse prevention, or through online peer support as described in other Web-based interventions for BD [10,11,13]. This study highlights the importance of the relationship that the users have with Web-based interventions and how this develops as an extension of the relationship with the humans perceived as offering and supporting its use. Web-based interventions offered in isolation in this context seem unlikely to engage people in the same way and may be perceived negatively as attempts to save money rather than improve care. Our study has explored the feasibility and acceptability of a specific Web-based intervention (ERPonline), but does not address the broader social issue of how acceptable the increasing use of digital health technology is to people with mental health problems [40].

Strengths of Study

Extensive user involvement improved the content of the ERPonline website, identified recruitment sources, and ensured the measures were appropriate and not too burdensome. The sample was sufficiently large to be able to comment on patterns in the data likely to be indicative of effects on process and outcome measures in a larger trial. Independent randomization, trained blind assessors, and the use of well-established outcome and process measures ensured that the data are reliable and valid. Extensive reflection and learning around feasibility was built into the design process using face-to-face meetings and an online reflection log.


Despite 280 unique site registrations, only 145 people consented, and due to ineligibility and drop out, only 96 were randomized. We have no data on why nearly half the sample registering an interest, then chose not to take part, though for some it may have been delay between prestart expression of interest and randomization. During the trial, we had higher dropout in the ERPonline arm, which is common in trials with a WL control arm and is likely due to the perceived reward of the intervention retaining people through WL. Survey responses were incomplete for feedback on trial participation (22/49 in WL group, 45%) and for feedback on the ERPonline intervention (17/47 in ERPonline arm, 36%). The bias in responders is likely to skew the nature of the feedback which on the whole was very positive.

In summary, we were able to successfully adapt and deliver online a relapse prevention intervention for BD previously used face-to-face. The intervention was successfully evaluated against a WL control group using a RCT design with high levels of retention and data completeness over 48 weeks. Participants had high rates of previous bipolar episodes but had accessed previous psychosocial interventions (where specified, most commonly described as CBT) for BD. Web-based interventions may prove an important cheap, feasible, and acceptable step forward in creating a choice of evidence-based interventions for people with BD at different stages of recovery, but may be more appropriately designed with built-in support and targeted at those with less prior experience of effective care.


The authors would like to thanks all the participants, the Service User Reference Group, and the Trial Steering Committee including Professor Danny Smith (Chair), Professor Gillian Haddock, Dr Kim Wright, Mr Tim Rawcliffe, and Dr Chris Sutton. This paper presents independent research commissioned by the National Institute of Health Research (NIHR) under the Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0211-10001). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. The funder played no role in the study design, conduct or interpretation of the data. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication. RM is funded by the Collaboration for Leadership in Applied Health Research and Care and East Midlands and the NIHR MindTech Health Technology Co-operative.

Authors' Contributions

FL was the study Chief Investigator, leading the design of the trial and ERPonline intervention, study conduct, and write up of the study. AD managed day-to-day management of the trial and data collection. OA and PD did the statistical analysis. AS, MG, and DK collected the data. MH analyzed the Web usage data. RL Chaired the SURG, ensuring continued service user input throughout the study. RP developed the ERPonline website. RM and SJ contributed to the design analysis and interpretation of data, and edited drafts of the paper. DD facilitated recruitment through NHS Trusts. All authors were involved in drafting and final approval of this paper.

Conflicts of Interest

ERPonline was developed by the authors and therefore this is not an independent evaluation. Prof Morriss is the Mood Disorders Theme Lead for the NIHR MindTech Health Technology Co-operative. He also chaired the NICE Clinical Guideline Development Group for bipolar disorder that recommended psychological treatment for long-term maintenance of bipolar disorder.

  1. Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013 Nov 09;382(9904):1575-1586. [CrossRef] [Medline]
  2. McCrone P. Paying the Price: The Cost of Mental Health Care in England to 2026. London: King's Fund Publishing; 2008.
  3. National Institute for HealthClinical Excellence. NICE. London: National Institute for Health and Care Excellence; 2014 Nov 24. Bipolar disorder: assessment and management (clinical guideline 185)   URL: https:/​/www.​​guidance/​cg185/​resources/​bipolar-disorder-assessment-and-management-35109814379461 [accessed 2017-03-02] [WebCite Cache]
  4. Malhi GS, Bassett D, Boyce P, Bryant R, Fitzgerald PB, Fritz K, et al. Royal Australian and New Zealand college of psychiatrists clinical practice guidelines for mood disorders. Aust N Z J Psychiatry 2015 Dec;49(12):1087-1206. [CrossRef] [Medline]
  5. Lehman AF, Lieberman JA, Dixon LB, McGlashan TH, Miller AL, Perkins DO, American Psychiatric Association, Steering Committee on Practice Guidelines. Practice guideline for the treatment of patients with schizophrenia, second edition. Am J Psychiatry 2004 Feb;161(2 Suppl):1-56. [Medline]
  6. Merikangas KR, Jin R, He J, Kessler RC, Lee S, Sampson NA, et al. Prevalence and correlates of bipolar spectrum disorder in the world mental health survey initiative. Arch Gen Psychiatry 2011 Mar;68(3):241-251 [FREE Full text] [CrossRef]
  7. Lobban F, Taylor L, Chandler C, Tyler E, Kinderman P, Kolamunnage-Dona R, et al. Enhanced relapse prevention for bipolar disorder by community mental health teams: cluster feasibility randomised trial. Br J Psychiatry 2010 Jan;196(1):59-63 [FREE Full text] [CrossRef] [Medline]
  8. Hollis C, Falconer C, Martin J, Whittington C, Stockton S, Glazebrook C, et al. J Child Psychol Psychiatr. 2016 Dec 10. Annual research review: digital health interventions for children and young people with mental health problems: a systematic and meta-review   URL: [accessed 2017-03-02] [WebCite Cache]
  9. Wallin EE, Mattsson S, Olsson EM. The preference for internet-based psychological interventions by individuals without past or current use of mental health treatment delivered online: a survey study with mixed-methods analysis. JMIR Ment Health 2016 Jun 14;3(2):e25 [FREE Full text] [CrossRef] [Medline]
  10. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One 2014;9(5):e98118 [FREE Full text] [CrossRef] [Medline]
  11. Drozd F, Vaskinn L, Bergsund H, Haga S, Slinning K, Bjørkli CA. The implementation of internet interventions for depression: a scoping review. J Med Internet Res 2016 Sep 08;18(9):e236 [FREE Full text] [CrossRef] [Medline]
  12. Smith DJ, Griffiths E, Poole R, di FA, Barnes E, Kelly MJ, et al. Beating bipolar: exploratory trial of a novel internet-based psychoeducational treatment for bipolar disorder. Bipolar Disord 2011;13(5-6):571-577. [CrossRef] [Medline]
  13. Murray G, Leitan ND, Berk M, Thomas N, Michalak E, Berk L, et al. Online mindfulness-based intervention for late-stage bipolar disorder: pilot evidence for feasibility and effectiveness. J Affect Disord 2015 Jun 01;178:46-51. [CrossRef] [Medline]
  14. Lauder S, Chester A, Castle D, Dodd S, Gliddon E, Berk L, et al. A randomized head to head trial of an Internet based self-help program for bipolar disorder. J Affect Disord 2015 Jan 15;171:13-21. [CrossRef] [Medline]
  15. Proudfoot J, Parker G, Manicavasagar V, Hadzi-Pavlovic D, Whitton A, Nicholas J, et al. Effects of adjunctive peer support on perceptions of illness control and understanding in an online psychoeducation program for bipolar disorder: a randomised controlled trial. J Affect Disord 2012 Dec 15;142(1-3):98-105. [CrossRef] [Medline]
  16. Barnes C, Hadzi-Pavlovic D, Wilhelm K, Mitchell P. A web-based preventive intervention program for bipolar disorder: outcome of a 12-months randomized controlled trial. J Affect Disord 2015 Mar 15;174:485-492. [CrossRef] [Medline]
  17. Simon GE, Ludman EJ, Goodale LC, Dykstra DM, Stone E, Cutsogeorge D, et al. An online recovery plan program: can peer coaching increase participation? Psychiatr Serv 2011 Jun;62(6):666-669 [FREE Full text] [CrossRef] [Medline]
  18. Todd NJ, Jones SH, Hart A, Lobban FA. A web-based self-management intervention for Bipolar Disorder 'living with bipolar': a feasibility randomised controlled trial. J Affect Disord 2014 Dec;169:21-29. [CrossRef] [Medline]
  19. Bauer R, Conell J, Glenn T, Alda M, Ardau R, Baune BT, et al. Internet use by patients with bipolar disorder: results from an international multisite survey. Psychiatry Res 2016 Aug 30;242:388-394. [CrossRef] [Medline]
  20. Lobban F, Dodd AL, Dagnan D, Diggle PJ, Griffiths M, Hollingsworth B, et al. Feasibility and acceptability of web-based enhanced relapse prevention for bipolar disorder (ERPonline): trial protocol. Contemp Clin Trials 2015 Mar;41:100-109 [FREE Full text] [CrossRef] [Medline]
  21. First M. User's Guide for the Structured Clinical Interview for DSM-IV Axis I Disorders SCID-I: Clinician Version. Washington, DC: American Psychiatric Press; 1997.
  22. Lobban F, Solis-Trapala I, Symes W, Morriss R, ERP Group‚ University of Liverpool. Early warning signs checklists for relapse in bipolar depression and mania: utility, reliability and validity. J Affect Disord 2011 Oct;133(3):413-422. [CrossRef] [Medline]
  23. Lobban F, Solis-Trapala I, Tyler E, Chandler C, Morriss RK. The role of beliefs about mood swings in determining outcome in bipolar disorder. Cogn Ther Res 2012 Mar 29;37(1):51-60. [CrossRef]
  24. Thompson K, Kulkarni J, Sergejew AA. Reliability and validity of a new medication adherence rating scale (MARS) for the psychoses. Schizophr Res 2000 May 05;42(3):241-247. [Medline]
  25. Keller MB, Lavori PW, Friedman B, Nielsen E, Endicott J, McDonald-Scott P, et al. The longitudinal interval follow-up evaluation. a comprehensive method for assessing outcome in prospective longitudinal studies. Arch Gen Psychiatry 1987 Jun;44(6):540-548. [Medline]
  26. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry 1960 Feb;23:56-62 [FREE Full text] [Medline]
  27. Bech P, Rafaelsen OJ, Kramp P, Bolwig TG. The mania rating scale: scale construction and inter-observer agreement. Neuropharmacology 1978 Jun;17(6):430-431. [Medline]
  28. Morosini PL, Magliano L, Brambilla L, Ugolini S, Pioli R. Development, reliability and acceptability of a new version of the DSM-IV social and occupational functioning assessment scale (SOFAS) to assess routine social functioning. Acta Psychiatr Scand 2000 Apr;101(4):323-329. [Medline]
  29. Jaeger J, Berns SM, Czobor P. The multidimensional scale of independent functioning: a new instrument for measuring functional disability in psychiatric populations. Schizophr Bull 2003;29(1):153-168. [Medline]
  30. Mundt JC, Marks IM, Shear MK, Greist JH. The work and social adjustment scale: a simple measure of impairment in functioning. Br J Psychiatry 2002 May;180:461-464 [FREE Full text] [Medline]
  31. Michalak EE, Murray G, Collaborative RESearch Team to Study Psychosocial Issues in Bipolar Disorder (CREST.BD). Development of the QoL.BD: a disorder-specific scale to assess quality of life in bipolar disorder. Bipolar Disord 2010 Nov;12(7):727-740. [CrossRef] [Medline]
  32. Jones S, Mulligan LD, Higginson S, Dunn G, Morrison AP. The bipolar recovery questionnaire: psychometric properties of a quantitative measure of recovery experiences in bipolar disorder. J Affect Disord 2013 May;147(1-3):34-43. [CrossRef] [Medline]
  33. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011 Dec;20(10):1727-1736 [FREE Full text] [CrossRef] [Medline]
  34. Chisholm D, Knapp MR, Knudsen HC, Amaddeo F, Gaite L, van WB. Client socio-demographic and service receipt inventory--European version: development of an instrument for international research. Epsilon study 5. European psychiatric services: inputs linked to outcome domains and needs. Br J Psychiatry Suppl 2000(39):s28-s33. [Medline]
  35. Ritchie J, Lewis J. Qualitative Research Practice: A Guide for Social Science Students and Researchers. London: Sage Publications; 2003.
  36. Morgan AJ, Jorm AF, Mackinnon AJ. Internet-based recruitment to a depression prevention intervention: lessons from the Mood Memos study. J Med Internet Res 2013 Feb 12;15(2):e31 [FREE Full text] [CrossRef] [Medline]
  37. Oud M, Mayo-Wilson E, Braidwood R, Schulte P, Jones SH, Morriss R, et al. Psychological interventions for adults with bipolar disorder: systematic review and meta-analysis. Br J Psychiatry 2016 Mar;208(3):213-222. [CrossRef] [Medline]
  38. Hajebi A, Motevalian A, Amin-Esmaeili M, Hefazi M, Radgoodarzi R, Rahimi-Movaghar A, et al. Telephone versus face-to-face administration of the structured clinical interview for diagnostic and statistical manual of mental disorders, fourth edition, for diagnosis of psychotic disorders. Compr Psychiatry 2012 Jul;53(5):579-583. [CrossRef] [Medline]
  39. Jones S, Wainwright LD, Jovanoska J, Vincent H, Diggle PJ, Calam R, et al. An exploratory randomised controlled trial of a web-based integrated bipolar parenting intervention (IBPI) for bipolar parents of young children (aged 3-10). BMC Psychiatry 2015 Jun 06;15:122 [FREE Full text] [CrossRef] [Medline]
  40. Poder TG, Bellemare CA, Bédard SK, Lemieux R. Social acceptance and population confidence in telehealth in Quebec. BMC Health Serv Res 2015;15:72 [FREE Full text] [CrossRef] [Medline]

BBC: British Broadcasting Corporation
BD: bipolar disorder
BIPQ: Brief Illness Perception Questionnaire
BRQ: Bipolar Recovery Questionnair
CBT: cognitive behavior therapy
CSE: Certificate of Secondary Education
CSRI: Client Service Receipt Inventory
CTU: clinical trials unit
DSM-IV: Diagnostic and Statistical Manual of Mental Disorders-IV
ERP: enhanced relapse prevention
EWS: early warning signs
GCSE: General Certificate of Secondary Education
HAM-D: Hamilton Depression Rating Scale
HR: hazard ratio
MARS: Medication Adherence Rating Scale
MRS: Mania Rating Scale
MSIF: Multidimensional Scale of Independent Functioning
NHS: National Health Service
NICE: National Institute for Health and Care Excellence
PG: postgraduate
PSP: Personal and Social Performance Scale
QoLBD: Quality of Life in Bipolar Disorder
RA: Research Associate
RCT: randomized controlled trial
SAE: severe adverse events
SCID: Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV)
TSC: Trial Steering Committee
WL: waitlist
WSAS: Work and Social Adjustment Scale

Edited by G Eysenbach; submitted 17.11.16; peer-reviewed by A Sharma, T Poder; comments to author 22.12.16; revised version received 18.01.17; accepted 11.02.17; published 24.03.17


©Fiona Lobban, Alyson L Dodd, Adam P Sawczuk, Ozgur Asar, Dave Dagnan, Peter J Diggle, Martin Griffiths, Mahsa Honary, Dawn Knowles, Rita Long, Richard Morriss, Rob Parker, Steven Jones. Originally published in the Journal of Medical Internet Research (, 24.03.2017.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.