Consumers with lived experience of BD were involved in all phases of program development, aligning with the consumer-based participatory research approach [14]. Consumer feedback from the pilot phase [8] guided content development for the current iteration of ORBIT. Intervention content is largely driven by the consumer voice, captured via videos of those with lived experience of BD. A local consumer advisory group (CAG) was established for the ORBIT project, comprising 10 individuals (6 females and 4 males) diagnosed with BD. Three CAG meetings took place during the development phase. Members played an integral role in providing feedback on content and the website itself (look-and-feel, ease of use, any perceived benefits or barriers) to ensure appropriateness and relevance to those with BD. We additionally consulted our pre-established CAG (located in Canada), part of the Collaborative Research Team to study psychosocial issues in BD (CREST.BD), to ensure that intervention content was appropriate for a broader international audience.

As part of the RCT, qualitative feedback from participants completing the program was collected to (1) guide future developments and (2) provide insights into their level of engagement with the Web-based intervention. These practices extend beyond the usual application of face-to-face intervention for the Web-based format by allowing a largely consumer-driven process. This approach ensures the program is tailored to the population it seeks to serve and aligns with the recovery model focus of empowering consumers via their active involvement in all stages of intervention development and testing.

Our international team of researchers, clinicians, and consumers initially developed and piloted program content drawn from mindfulness-based therapies [8]. The program was then extensively revised and extended, iteratively developed via a dynamic interplay of theoretically derived therapeutic content and footage from videos of consumers with lived experience of BD. Content for Web-based interventions is often adapted from a face-to-face psychological intervention for Web-based delivery, as part of a one-step process where the content is largely defined upfront. By contrast, ORBIT content was progressively developed over a 6-month period. The process commenced with the “top-down” development of a semistructured interview schedule, drawing from broad topic areas aligned with mindfulness-based therapeutic approaches and BD psychoeducation. Clinical psychologists and mindfulness practitioners were consulted on topic areas to ensure that questions were grounded in a psychological framework. Next, 12 consumers were recruited to participate in the filming process: consumers were selected on the basis of gender; age; cultural background; and a range of experiences with mindfulness, acceptance-based approaches and self-management strategies (eg, recognizing early warning signs and triggers), ensuring a representative sample that ORBIT participants could relate to. The video process adopted a documentary-style interview, lasting up to 2 hours per person, undertaken with a professional film crew. Consumers were encouraged to speak from personal experience to ensure footage captured the central “consumer voice.” This “bottom-up” generation of program content was balanced within the “top-down” psychological framework—footage was carefully reviewed, and the project team iteratively revisited the planned content and structure of the intervention to maintain this balance. Hours of footage were edited into “snippets” from a range of consumers and combined into short videos (3 minutes on average in length), describing particular skills and experiences. This new type of delivery (documentary-style videos as an engagement strategy for Web-based interventions) ultimately led to a novel way of developing a Web-based intervention and, ultimately, a new intervention. The final stage of development involved fleshing out content from videos, including key summary messages, supplementary text, and MP3 audio files (allowing for in-the-moment experiential practice of newly introduced concepts) to promote learning and encourage skill development. Program content was then reviewed by our local and international CAGs and revised on the basis of their feedback. Overall, feedback indicated that videos successfully captured the consumer voice, providing a “real-world” feel that people could relate to, adding credibility to the program (balancing realistic messages with hope), reducing stigma, and allowing subtle processes to be described in an engaging and personalized way:

CAG members were consulted on the duration of videos; in general, there was agreement that “shorter” (3-4 minutes) videos were preferable in terms of maintaining interest and engagement, without being too cognitively demanding. This is in accordance with our prior experience in developing video-based content for Web-based interventions for individuals with persisting psychosis as a strategy to promote recovery [15]. Finally, the use of informational PDFs, clinician videos, and instructional videos (eg, consumer or expert walk-through, how to use a mood tracking tool) allows concepts to be easily understood within a user-friendly context. The Web-based environment lends itself to participants revisiting content as needed, at their own pace, to consolidate their understanding and facilitate repeated practice.

Content is guided by psychological principles; thus, consideration of how this can be made engaging, interactive, flexible, and appealing is essential to the success of any Web-based self-management program. Given the cognitive impairments common in those with BD, practical considerations (eg, being able to start, pause, and recommence topic areas; videos of short duration accompanied by transcripts; length of exercises; language; and neutral colors and icons) were carefully considered to ensure ongoing engagement.

The programs were designed to maximize constructive engagement by participants having access to a Web-based community (other participants, moderated forums), allowing them to link in with social support that may not otherwise have been realized [16]. This can provide a sense of normalization and a way in which they can also support others. The forum facilitates sharing of key learnings as participants navigate through the program, and a secure-messaging system allows participants to foster connections with other users if they so wish (extending social support networks outside of ORBIT). Participants are assigned a Web-based coach as part of the ORBIT community, with asynchronous message support. As overviewed by others [17], support in Web-based interventions for BD and other psychological conditions reduces dropout rates, making them comparable to rates observed in face-to-face therapies. Indeed, motivation to persist with Web-based interventions has been found when needs for relatedness (eg, identifying with other end users and content, support from Web-based coach, forum participation) are satisfied [18].

Our group has been interested in developing novel psychotherapies for BD, drawing on third-wave principles of mindfulness and self-compassion [19]. There have been anecdotal concerns in the literature about potential iatrogenic effects of mindfulness for people with BD [8]. One of the arms in the ORBIT project contains such elements, requiring meticulous attention to ethical issues and risk management. Current mood state is an important factor to consider when introducing those with BD to experiential techniques within a Web-based intervention, given the risk of triggering mood dysregulation. Clinical caution messages (both within the content and incorporated into audio exercises) are used to empower participants to consider whether practicing a particular technique at that moment would be beneficial or should be delayed. For instance, anecdotal reports of body scan exercises that are lengthy (eg, 20 minutes or longer) indicate they can be triggering (of mood episodes) for some; thus, cautions based on current mood state can be particularly useful (Figure 2).

This type of encouragement allows participants to self-pace through the program, allowing for increased engagement and a sense of autonomy. Participants are encouraged to reflect on exercises, and any adverse events arising that are directly related to intervention content are logged.

A widely recognized problem in internet delivery is transitioning from research-funded development and trialing of an intervention to long-term sustainable delivery outside a research funding environment. Swinburne University of Technology has been at the forefront of improving the sustainability of Web-based interventions by advocating (successfully) for the national recognition of eTherapy hours as integral to the training of Clinical Psychology Masters students (25 eTherapy hours approved by the Australian Psychological Society College of Clinical Psychologists). In the ORBIT trial, and in future potential iterations of the ORBIT website beyond the research phase, the primary personnel support (coaching) is provided by Clinical Psychology Masters students as part of their first internal placement at the university, a model that can be generalized to any psychological interventions offered in Australia.

A key facet of minimizing bias in treatment trials is ensuring that participants are randomly allocated to comparison conditions [20]. Web-based RCTs benefit from the use of fully automated computer-generated blocked randomization and allocation methods, ensuring all aspects of randomization are fully concealed to research personnel. Such methods are superior to traditional RCT methods (eg, sequentially numbered opaque sealed envelopes, central randomization by telephone to a trials office, etc) as no human involvement is required (an airtight method of concealment, reducing bias) and time efficiency is achieved (randomization and allocation can occur within seconds of each other, allowing participants to commence the intervention almost immediately). The latter is particularly important for studies involving those with frequent mood changes (ie, BD) that can occur within hours or days, impacting study data.

A central challenge posed by the Web-based context is how best to define intervention “dose.” An effective “dose” can constitute the level of usage needed for participants to benefit from the use of the program, the extent that the “dose” varies between participants, and participant characteristics that may influence the “dose” that is needed [21].

Participants differ in Web-based usage patterns, with evidence suggesting that less time will be spent on the program than researchers expect [22]. Program usage statistics automatically recorded onsite (eg, number of pages viewed, number of tasks completed, timestamps, etc) are an essential measure of adherence and engagement, providing an objective proxy for “dose” received. Study investigators need to identify program usage statistics of interest during the development stage to communicate this to the website developer (who will incorporate selected variables into data downloads for later analysis). Usage data can automatically be tracked and operationalized in an algorithm combining time on the Web, activity completion, and active engagements with the intervention [23]. However, adherence (and the related construct of engagement) is difficult to characterize and measure—for instance, the proportion of time spent on a particular page does not necessary represent the time participants were actively engaged on that page (they may have been chatting on Facebook or away from their computer). Furthermore, adherence alone does not capture a participant’s entire experience of an intervention [23]; there is growing interest in understanding other ways in which individuals engage with Web-based interventions, and the meaning of engagement in this context [24]. Currently, minimal consensus exists on the definition and conceptualization of engagement with Web-based interventions. Some view engagement as synonymous with adherence and the opposite of “intervention attrition” or “treatment dropout” [25,26], while others consider it to move beyond mere attendance, incorporating the extent that an individual actively participates in a treatment on offer behaviorally, cognitively, and affectively [27,28].

To develop a more comprehensive understanding of adherence and engagement, the ORBIT project will examine both objective and subjective levels of “active” participation; usage statistics will be captured via the website, and in-depth qualitative interviews will be conducted with a subset of participants asking about their usage (online and “offline” practice of skills) and perceived level of engagement with the content. Through this process, we will develop an innovative algorithm to quantify the important variables of “minimum dose” and “attrition” in the Web-based context for inclusion in statistical analyses. As discussed in the protocol paper [9], the concept of attrition appears in two sensitivity analyses of the primary outcome. First, per-protocol analyses will be conducted on those receiving a minimal dose of the intervention, with minimal dose to be defined on the basis of the pending algorithm of self-reported and automatically recorded usage variables. Second, intention-to-treat analyses will be repeated with imputation on baseline and relevant postrandomization variables; these variables could include attrition (again defined in the pending algorithm of usage variables) if it is shown to differ between groups. As also outlined in the protocol paper, we will attempt to follow and assess all participants regardless of the level of usage of the site during the intervention period (with the exception of those explicitly discontinuing or being withdrawn on ethical grounds).

As outlined in our protocol paper [9], the interventions follow best practice in persuasive system design. Three key features known to impact on engagement are utilized: (1) dialogue support (praise from coach and forum moderator, email reminders); (2) social support (social facilitation through discussion threads in moderated forums); and (3) primary task support (best-practice principles for modularization of content, personalization or monitoring of progress, prompted self-monitoring, and rehearsal) [29]. Furthermore, the intervention is brief, and program content is released to each participant sequentially each week in an attempt to pace users as they work their way through the program, while gradually building on knowledge from earlier weeks (with “teaser” messages with respect to upcoming content). We opted against the delivery strategy of a single exposure (receipt of all content in one go), despite some evidence suggesting engagement rates increase when end users have control over how they view content, along with free choice on when they interact with it [30]. Our decision was guided by qualitative consumer feedback from our pilot study [8], indicating some end users felt overwhelmed with the amount of content.

An open approach to navigation within each week’s content invites a further challenge—the program is essentially a “set of offerings” rather than “sessions” or “modules” to be completed, catering where possible for different users (eg, those with no experience of mindfulness vs regular meditators; those who have limited time to spend on the program). Unlike manualized treatment programs adapted for the Web-based format, one cannot assume that participants will work their way through each week’s content in a sequential manner. This required careful consideration during ORBIT content development, with persuasive technologies (eg, links and “suggestions” embedded within the content) used to prompt participants on how they might best navigate their way through the program, “chunking” of material to allow for shorter sessions times depending on user preferences, and suggestions for skills practice peppered throughout the content in case users did not click through to the “homework” page located at the end of each week’s content. Topic areas were crafted in a largely self-contained way, for example, using mindfulness (ranging from beginners exercises to more advanced practices) as a guiding overall skill to integrate topics (Figure 3).

Consumers (referred to as “superusers”) with lived experience of BD were employed for the ORBIT project and trained to assist with seeding forum content and facilitating engagement with the content. This strategy brings unique opportunities and challenges. While a rich and lively forum community has developed and participants appreciate and benefit from communication with peers, guiding principles have been developed to manage the ethical and scientific impact of this dynamic environment. The superuser role is defined as nontherapeutic, and a degree of self-disclosure is encouraged.

From a scientific perspective, superusers are instructed to seed forum content with messages that are specific to the program being offered (eg, mindfulness-related material), avoiding (where possible) cross-contamination with content from the alternate program. Monthly supervision of superusers is undertaken to ensure that these guiding principles are adhered to while balancing the tension of allowing the forums to organically unfold as part of a dynamic Web-based community.

Delivering an adjunctive Web-based self-management program for BD (and indeed, any chronic mental health condition with substantial clinical risks) within the context of an international research trial brings a level of risk that requires clear protocols should adverse events arise. Real-time intervention is not always possible or feasible given users are on the Web during different time zones, and its use must be carefully considered.

Motivated by clinical risk-management priorities, as well as principles around generalization of learning through participation, an approach was developed that emphasizes participants’ local networks of treatment and care [31]. Specifically, our approach was to explicitly devolve safety and well-being to participants and their local network (treating mental health practitioner and local emergency services). This is achieved in three key ways. First, an inclusion criterion requires participants to have had contact with a mental health professional during the past 12 months, to provide these details to the project team, agree for this professional to be contacted if necessary, and understand that they remain the first point of contact. Mental health professionals are posted a courtesy letter informing them of their clients’ participation in the ORBIT project. Second, participants are explicitly made aware (via the consent form, during the sign-up process when speaking to the research team, information on the program websites, coach messages, and forum messages from superusers) that the program is not intended to replace their usual care, does not provide a crisis service, and is not monitored in real-time. Links to emergency resources are, nonetheless, provided on the program websites (eg, unsuicide.wikispaces.com). Participants are directed back to their mental health professional as needed throughout their involvement in the ORBIT project. Third, a “red flag” protocol was developed to guide adverse event procedures, based on our experience with other Web-based interventions and websites for BD [32-37] and consultation with the CREST.BD CAG. This is detailed in the protocol paper [9]. In essence, the decision tree distinguishes between red flag information (which can arise during research assessments, Web-based questionnaires, forum posts, and messages to coach), suggesting immediate risk of harm for which real-time intervention is feasible or recommended (eg, when active suicidality is identified during a phone assessment), and when it is not (eg, when the research team becomes aware of active suicidality mentioned in a forum post from 48 hours previous). Actions are progressively escalated to senior staff members if required. As an example, forums are moderated by the project manager; inappropriate content is initially flagged (both in terms of risk and potential for triggering other participants) by superusers, prompting appropriate actions. All trial staff are comprehensively trained on these protocol procedures, operating to the guiding principle that participants’ local treatment and care networks are not disrupted. As per standard ethical guidelines, participants are withdrawn from the ORBIT project on a case-by-case basis should it be deemed that their well-being is compromised by their participation, and serious adverse events suspected or known to be related to participation in the trial are reported to the local administering ethics committee (Swinburne University of Technology Human Research Ethics Committee).

Related to risk management, participant distress (whether arising as a direct result of participation in the project or as part of the usual clinical course of the mental health condition) is an ongoing challenge and particularly so for remotely delivered interventions. As for the majority of RCTs (whether Web-based or face-to-face), the ORBIT project includes a structured diagnostic interview, the Mini-International Neuropsychiatric Interview [38] to assess inclusion or exclusion criteria. Structured diagnostic interviews are, by their very nature, highly detailed and require participants to revisit their experiences of distressing symptoms. When conducted face-to-face, interviewers are able to pay attention to nonverbal cues and can manage distress levels sensitively. This poses a particular challenge for interviews conducted over the phone, as for the ORBIT project. Distress associated with participation in the diagnostic interview has been flagged by consumers participating in training for research staff administering the Mini-International Neuropsychiatric Interview, as well as a small proportion of ORBIT project participants. The detailed nature and duration of the baseline interview (lasting up to 2 hours) can be particularly distressing for those who have experienced multiple mood episodes. Solutions to date have included warning participants upfront of the potential for distress arising, interviewers being trained to tune in to participant tone of voice and other verbal cues, offering frequent breaks during the interview and conducting the interview across a few sessions if needed. While structured diagnostic interviews are a standard component of mental health research, sensitivity to participant distress and burden (particularly for those with chronic disorders such as BD) is a key priority. Offering participants the opportunity to debrief following such interviews may be an additional solution for future studies in this space.

A key issue currently facing the delivery of Web-based “interventions” for mental health concerns the type of intervention that is offered. Interventions claiming to have some therapeutic value (eg, psychological interventions) fall into a gray legal zone, whereby certain jurisdictions require the “therapist” providing the “intervention” to be registered in the state, territory, or country where the client accesses the service from. Indeed, some states could potentially prosecute the remote “therapist” under the state’s laws. As clients can access Web-based programs from any location worldwide, this presents a legal minefield. As a first step to navigate this, the terminology used to describe the intervention must be carefully specified—for example, the ORBIT project does not claim to offer a psychological service, rather a self-management program that complements (but does not replace) usual clinical care. Second, care should be taken in defining what the program is intended to offer—the ORBIT project indicates that improvements in quality of life may result from completing the program. Third, as a guided program where participants have access to a personal coach, coach qualifications and role are made clear upfront. Specifically, the coaching role is nontherapeutic, with the key aim of supporting participants in terms of their engagement with program content. The tone and content of coach messages are carefully crafted on the basis of general guiding principles to standardize responses where possible (while remaining “human”), and coaches receive regular supervision. For example, responses tending toward therapeutic statements (eg, advanced empathic insights) were discouraged on two grounds: (1) the asynchronous email communication cannot sustain such a dialogue and (2) these could weaken participants’ engagement with their own local therapeutic resources. Finally, as a Web-based program within the context of an RCT, we strategically positioned the ORBIT project as a “single-site” study, governed by a single Human Research Ethics Committee (HREC; in this case, Swinburne HREC), offering services delivered from Australia outlined in a disclaimer present in the website terms and conditions reading, “The services on the ORBIT website are provided in accordance with Australian laws and health practice standards. You acknowledge and accept that the services may not comply with the laws and standards that apply in the jurisdiction in which you receive the services. ” Many of the legal issues associated with the delivery of Web-based interventions remain unknown at present while university HRECs have not developed internal processes to deal with such issues.

Insights across the three overlapping domains (intervention development, scientific testing, and ethical frameworks) are now summarized. Web-based interventions can be optimized through (1) codesign with consumers with lived experience to ensure relevance and appropriateness to the target audience; (2) novel content development processes that iteratively combine evidence-based information with lived experience perspectives, capitalizing on multimedia (eg, videos) that the digital health space provides; (3) incorporating Web-based communities to connect end users and promote constructive engagement via access to a Web-based coach. The potential iatrogenic effects of particular exercises (eg, those of an experiential nature) within the intervention content must be considered for the target group of interest, with clinical caution messages and self-pacing encouraged to ensure end users move safely (and autonomously) through program content. Finally, sustainability models that are generalizable (eg, personnel support for guided interventions outside of an RCT context) should be considered as part of the development process.

Within the scientific context, while Web-based RCTs offer swift, unbiased randomization and allocation methods that can be fully automated, adherence and engagement with the intervention itself can be difficult to quantify. These concepts require further investigation; study designs should incorporate quantitative and qualitative assessment of adherence and engagement to move beyond automatically captured usage data and develop a richer understanding of how participants interact with Web-based interventions. These learnings can guide persuasive technologies to optimize an individual’s experience of the program. As a final method of engaging users, ORBIT seeks to build a dynamic Web-based community driven by consumers with lived experience. The ethical and scientific impact of this environment requires careful consideration; superuser roles require clear definition, seeded forum content should align with the intervention content (avoiding cross-contamination with the control condition), and ongoing supervision is required to allow forums to unfold organically while balancing scientific integrity.

Ethically, clear risk management protocols are required for Web-based self-management programs. Real-time intervention is not always possible, nor feasible, within the context of an international RCT. A guiding principle of the ORBIT project is to explicitly devolve safety and well-being to participants and their existing local care network, respecting clinical risk-management priorities and participant autonomy. While structured diagnostic interviews are central in determining inclusion criteria to RCTs, awareness of participant distress arising from such interviews requires sensitivity, careful training of clinical interviewers, and debriefing where necessary. Finally, careful consideration of legal issues surrounding the delivery of interventions claiming to have “therapeutic value” in different jurisdictions is warranted.

We offer some final insights with respect to the multifaceted skill set requirements for projects such as ORBIT that are broadly applicable to the development of any Web-based intervention. A level of technological expertise is needed to oversee the development, implementation, and evaluation of Web-based self-management programs. Expertise in this space can include knowledge of e-learning, e-communication, health informatics, basic programming skills, and awareness of technological barriers that could deter use (eg, slow internet speed and ability to watch videos), with the overall aim of ensuring that the program is innovative, engaging, feasible, and likely to be effective. As described by others [39], the ability of mental health researchers to enter the world of the Web-based program developer invites valuable opportunities to influence scoping, design, and evaluation. This offers a new skill set to mental health researchers—a new “breed of transdisciplinary experts” [39]—allowing highly innovative and clinically effective electronic mental health programs to be developed.