Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 13.08.20 in Vol 22, No 8 (2020): August

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/17155, first published Nov 21, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    A Novel Multimodal Digital Service (Moderated Online Social Therapy+) for Help-Seeking Young People Experiencing Mental Ill-Health: Pilot Evaluation Within a National Youth E-Mental Health Service

    1Orygen, Parkville, Australia

    2Centre for Youth Mental Health, The University of Melbourne, Melbourne, Australia

    3The School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia

    4headspace National Youth Mental Health Foundation, Melbourne, Australia

    5School of Culture and Communications, The University of Melbourne, Melbourne, Australia

    6Centre for Mental Health, Swinburne University of Technology, Hawthorn, Australia

    7Hotchkiss Brain Institute, Psychiatry Department, University of Calgary, Calgary, AB, Canada

    8Consorci Sanitari del Maresme, Department of Psychiatry, Hospital of Mataro, Mataro, Spain

    9School of Behavioural and Health Sciences, Australian Catholic University, Melbourne, Australia

    10Healthy Brain and Mind Research Centre, Australian Catholic University, Melbourne, Australia

    Corresponding Author:

    Mario Alvarez-Jimenez, BSc(Hons), DClinPsy, MAResearchMeth, PhD

    Orygen

    35 Poplar Rd

    Parkville, 3052

    Australia

    Phone: 61 9966 9383

    Email: mario.alvarez@orygen.org.au


    ABSTRACT

    Background: Mental ill-health is the leading cause of disability worldwide. Moreover, 75% of mental health conditions emerge between the ages of 12 and 25 years. Unfortunately, due to lack of resources and limited engagement with services, a majority of young people affected by mental ill-health do not access evidence-based support. To address this gap, our team has developed a multimodal, scalable digital mental health service (Enhanced Moderated Online Social Therapy [MOST+]) merging real-time, clinician-delivered web chat counseling; interactive user-directed online therapy; expert and peer moderation; and peer-to-peer social networking.

    Objective: The primary aim of this study is to ascertain the feasibility, acceptability, and safety of MOST+. The secondary aims are to assess pre-post changes in clinical, psychosocial, and well-being outcomes and to explore the correlations between system use, perceived helpfulness, and secondary outcome variables.

    Methods: Overall, 157 young people seeking help from a national youth e-mental health service were recruited over 5 weeks. MOST+ was active for 9 weeks. All participants had access to interactive online therapy and integrated web chat counseling. Additional access to peer-to-peer social networking was granted to 73 participants (46.5%) for whom it was deemed safe. The intervention was evaluated via an uncontrolled single-group study.

    Results: Overall, 93 participants completed the follow-up assessment. Most participants had moderate (52/157, 33%) to severe (96/157, 61%) mental health conditions. All a priori feasibility, acceptability, and safety criteria were met. Participants provided mean scores of ≥3.5 (out of 5) on ease of use (mean 3.7, SD 1.1), relevancy (mean 3.9, SD 1.0), helpfulness (mean 3.5, SD 0.9), and overall experience (mean 3.9, SD 0.8). Moreover, 98% (91/93) of participants reported a positive experience using MOST+, 82% (70/93) reported that using MOST+ helped them feel better, 86% (76/93) felt more socially connected using it, and 92% (86/93) said they would recommend it to others. No serious adverse events or inappropriate use were detected, and 97% (90/93) of participants reported feeling safe. There were statistically significant improvements in 8 of the 11 secondary outcomes assessed: psychological distress (d=−0.39; P<.001), perceived stress (d=−0.44; P<.001), psychological well-being (d=0.51; P<.001), depression (d=−0.29; P<.001), loneliness (d=−0.23; P=.04), social support (d=0.30; P<.001), autonomy (d=0.36; P=.001), and self-competence (d=0.30; P<.001). There were significant correlations between system use, perceived helpfulness, and a number of secondary outcome variables.

    Conclusions: MOST+ is a feasible, acceptable, and safe online clinical service for young people with mental ill-health. The high level of perceived helpfulness, the significant improvements in secondary outcomes, and the correlations between indicators of system use and secondary outcome variables provide initial support for the therapeutic potential of MOST+. MOST+ is a promising and scalable platform to deliver standalone e-mental health services as well as enhance the growing international network of face-to-face youth mental health services.

    J Med Internet Res 2020;22(8):e17155

    doi:10.2196/17155

    KEYWORDS



    Introduction

    Background

    Mental ill-health is the number one cause of disability worldwide [1] and accounts for 8 million deaths annually [2]. Mental illness strikes the young, with 75% of all mental disorders having their onset before the age of 25 years [3], and for many, it follows a relapsing course [4,5]. The timing and course of mental illness disrupts the attainment of critical developmental milestones for young people, such as completing their education, entering the job market, and developing intimate relationships [6]. This can result in devastating lifelong consequences, including increased risk of chronic unemployment, lower income and living standards, homelessness, social isolation, suicide, and early mortality [7-9].

    Despite the prevalence and impact of mental illness, between 35% and 57% of people with mental health disorders do not access treatment in high-income countries [1]. The corresponding range for low- and middle-income countries is 76% to 85% [1]. The mismatch between the prevalence of mental illness and the rate at which treatment is accessed is the greatest for young people. The most recent Australian national survey of young people’s mental health revealed that only 13% of men and 31.2% of women aged 16-24 years who had experienced a mental disorder in the preceding 12 months received professional help [10]. The reasons for low rates of treatment access among young people include low help-seeking due to fear of stigma and embarrassment, confidentiality concerns, negative prior experiences of treatment, poor mental health literacy, lack of knowledge of available resources, a preference for self-reliance, and social isolation [11-13]. Additional structural and logistical barriers include geographical distance, poverty, out-of-pocket expenses associated with treatment, and lack of availability of services [14].

    The internet, mobile technologies, and social media have the potential to address the global crisis in the rate at which young people access evidence-based mental health care. Internet-enabled mobile devices are a pervasive element of young people’s lives, with 45% of adolescents being on the web almost constantly [15]. Social media has become a key vehicle for young people to communicate with one another. Almost all young people have at least one active social media account, with over 70% using social media multiple times a day—a rate that has doubled between 2012 and 2018 [16]. Recent surveys demonstrate that young people experiencing mental ill-health are also avid users of social media: 97% use it regularly, 2.6 hours per day on average [17]. Particularly relevant to the clinical potential of social media, engaging with peers online about mental health issues increases the likelihood of seeking professional support [18], and many young people use social media to obtain mental health information [19]. Similarly, initial studies showed that 74% of young people who experience mental ill-health would like to obtain help from mental health clinicians via social media [17] and value web-based services run by professionals [20]. Thus, social media offers a unique opportunity to provide and boost web-based youth mental health interventions.

    Currently, there are 4 main types of digital interventions for mental health: self-guided web-based interventions, standalone mental health mobile apps, online peer support groups or interventions, and web-based counseling with registered professionals. Previous trials have shown that the first generation of self-guided web-based interventions, particularly those delivering cognitive behavioral therapy (CBT) and including human support [21], are effective in alleviating anxiety and depressive symptoms [22,23]. However, the impact of these web-based interventions has been hindered by two constraints. First, these interventions were designed to mimic face-to-face interventions, resulting in high attrition rates (particularly for interventions with no human support) and little treatment innovation (eg, novel intervention models harnessing mobile technology to provide intensive, real-time support) [24]. Second, these interventions were developed as separate products [24], with little consideration as to how they fit in with existing clinical services, resulting in an overall lack of integration of web-based support with clinical services [25].

    The number of mobile apps targeting mental health has grown exponentially over the past few years. According to a 2017 report, almost 500 unique apps were targeting mental health disorders in 2017 [26]. More recently, a systematic search of mobile apps focused on wellness and stress management found over 1000 publicly available apps [27]. However, although mental health apps are flooding the consumer market, very few studies have examined their effectiveness [28], and many apps do not follow evidence-based guidelines and principles [29]. For example, of the apps focused on psychosocial wellness and stress management, only 1% involved therapist support, and less than 2% were designed to supplement in-person treatment and 2% had any research publications (with most of these being a single feasibility or efficacy study) [27]. Examining the existing evidence, two meta-analyses found significant but small effects for reductions in anxiety [30] and depression [31] from smartphone interventions as compared with control conditions. However, a more recent meta-analysis including only mobile apps designed to treat mental health symptoms found small significant effects on depression, with no significant effects on anxiety, suicidal ideation, self-injury, or alcohol use [32]. It must be noted that the effects of mental health apps (g=−0.14 to 0.18) were significantly smaller than those of web-based mental health interventions [23,33-35] (g=0.84-1.09). These smaller effects may be explained by the lack of human support in most mental health apps; poor adherence, which is considered to be a major pitfall of mental health apps [36], or low use of key app features; and possibly a lack of sustained and structured attention to content and features that may be needed for an intervention to yield significant benefits [32]. To date, most mobile mental health apps have been designed in academic labs or by commercial companies, usually outside of clinical services, resulting in an overall lack of integration with existing clinical services [37].

    A growing number of online peer support groups and social networking sites (SNS) exist for people with mental health problems. Overall, the extant evidence suggests that online peer support groups can foster a sense of social connectedness, empowerment, and improved quality of life as well as reduce depression and emotional distress [38-40]. That said, the type and function of online peer support groups and SNS appear to have a significant effect on their outcomes. For example, unmoderated forums and SNS can lead to increased contagion, distress, and collusion among users [39]. Conversely, SNS interventions that have been moderated, ideally by professionals, have been found to be safe, engaging, supportive, and useful [39,41-44]. This is in keeping with the findings that young people with mental ill-health have a strong preference for purpose-built, moderated, social media–based interventions enabling peer-to-peer contact as well as clinician support [45]. Despite this emerging evidence, the majority of available peer support groups do not provide professional moderation and clinical support or incorporate evidence-based, user-directed web-based therapy.

    The third main type of digital support is web-based counseling (ie, real-time web chat with clinicians). There is initial evidence that web chat is an effective way to deliver mental health support [46], with recent trials showing comparable levels of effectiveness [47], therapeutic alliance [48], and perceived helpfulness with face-to-face therapy [48]. Recent studies also indicate that young people are increasingly turning to web chat to receive mental health support [49]. However, these web-based services are also constrained by capacity and scalability due to their reliance on one-to-one web-based support.

    Our group has developed a novel and evolving model of web-based behavioral interventions entitled Moderated Online Social Therapy (MOST). The MOST model merges (1) interactive web therapy, (2) peer-to-peer web-based social networking, (3) peer, and (4) clinical moderation. Successive iterations and evolutions of MOST have been successfully adapted for, and trialed with, young people with psychosis [42,50], at clinical risk of psychosis [43], suicidal risk [51], depression [44,52], and social anxiety [53], as well as relatives of young people with psychosis [54] and depression [55]. To respond to the crisis in access to care by young people who experience mental ill-health, our group has developed a new model of web-based clinical support entitled Enhanced Moderated Online Social Therapy (MOST+). MOST+ fully merges the effective components of web-based interventions, online peer support groups, and web-based counseling into a single platform. As such, MOST+ combines (1) a wide range of evidence-based, interactive, user-directed web-based interventions; (2) secure and supportive peer-to-peer web-based social networking; (3) peer moderator support; (4) clinical moderation; and (5) on demand web chat with registered clinicians. Thus, MOST+ was designed to deliver an accessible and scalable web-based mental health service catering to the varying needs and preferences of young people by flexibly integrating multiple modes of effective web-based support.

    Objectives

    The overarching aim of this study (trial registration: ACTRN12617000370303) was to determine the feasibility, acceptability, and safety of MOST+ for young people seeking online mental health support. The secondary aims of the project were (1) to assess changes in psychological distress, well-being, depression, stress, social support, loneliness, basic psychological needs (self-competence, relatedness, and autonomy), strengths usage, and mindfulness skills from the point of engagement to post intervention and (2) to explore the associations between system usage, perceived helpfulness, and secondary outcome variables. We hypothesized that MOST+ would be regularly used, favorably received, and safe against a priori established criteria (described in detail in the Results section) [56].


    Methods

    Study Design and Setting

    The methods of this study have been described in detail elsewhere [56]. Briefly, this study employed an uncontrolled single-group design [57]. The MOST+ intervention was embedded within eheadspace services, a national web counseling service funded by the federal government for young people aged 12 to 25 years in Australia [49]. eheadspace is staffed by qualified and supervised mental health clinicians providing synchronous web chat, email support, and telephone-based mental health support. For this study, 27 eheadspace clinicians were trained to deliver online support using the MOST+ platform. Two of these clinicians, based at the eheadspace operation center, provided web chat support via MOST+ from 4 PM to 12 AM every day. They were registered with mental health clinicians with prior specialist training and experience in the delivery of e-mental health support to young people in distress. The MOST+ platform was available for a period of 9 weeks, including a 5-week recruitment period (ACTRN12617000370303).

    Participants and Recruitment

    The sample included 157 young people recruited via an opt-in process at the point of entry to eheadspace through a link on the home page. The inclusion criteria were as follows: (1) help-seeking young people with concerns about their own mental health, (2) people aged 16 to 25 years, and (3) people with the ability to provide informed consent and comply with study procedures. The participants who indicated on the web that they understood and consented to the study procedures were recruited into the study.

    To ensure the safety of the online social network, some participants were excluded from access to the social networking component of MOST+ (defined as partial access; see under Web Counseling). These exclusions were (1) acute risk of self-harm requiring urgent intervention (ie, suicidal ideation with a current plan and intent) indicated by a young person on web chat or by endorsing predetermined screening questions; (2) participation in the MOST+ social network was deemed likely to interfere with appropriate clinical management of mental health symptoms (eg, psychosis) or increase the risk of harm to self or others by an eheadspace clinician; and (3) inability to confirm age and conduct induction via a research assistant telephone contact (Multimedia Appendix 1). Participants with partial access could access all other components of MOST+ (ie, web chat and interactive web-based therapy). This design feature enabled us to compare outcomes across these 2 levels of access.

    The mean age of the participants was 19.1 (SD 2.3) years, with 77% female participants. A total of 87% (137/157) of the participants were born in Australia and 11% (17/157) spoke languages other than English. Moreover, 70% (110/157) of the participants were from metropolitan areas, 28% (44/157) from rural areas, and 2% (3/157) from remote areas. In addition, 3% (5/157) of the participants identified themselves as Aboriginals and/or Torres Strait Islanders. Furthermore, 59% (93/157) of the participants had not previously used youth mental health services and 37% (58/157) had never received any mental health support. A total of 57% (89/157) of the participants were engaged in paid work and 77% (121/157) were studying part-time or full-time. The main reasons for seeking help included sadness (38%: 60/157) and anxiety (22%:35/157), followed by feelings of distress (9.6%:15/157). Baseline clinical measures indicated that the majority of participants had mental ill-health. Specifically, the mean baseline Kessler 10 (K10) score was 32.03 (SD 7.72), with 61% (57/93) scoring 30 (indicative of a severe mental health disorder) and 33% (31/93) scoring 25-29 (indicative of a moderate mental health disorder) [58] (Table 1). Similarly, the mean baseline Patient Health Questionnaire-9 (PHQ-9) score (measuring depression) was 15.76 (SD 6.32), with 59% (55/93) scoring 15 (indicative of moderately severe depression) and 24% (22/93) scoring 10-14 (indicative of moderate depression) [59] (Table 1 [60-71]).

    Table 1. Mean (SD) and within-group effect sizes (Cohen d) for outcome measures (N=93).
    View this table

    Intervention: MOST+

    A large multidisciplinary team of researchers, clinical psychologists, programmers, creative writers, graphic artists, and experts in human-computer interaction worked in collaboration with end users to iteratively develop the MOST+ platform [42,45]. MOST+ merged (1) interactive user-directed web-based therapy (Steps), (2) peer-to-peer online social networking, (3) peer moderator support, (4) expert moderation, and (5) on demand web chat with clinicians.

    MOST+ was conceived as an accessible web-based youth mental health service delivering immediate, short-term, flexible, and evidence-based support to help-seeking young people with mental ill-health. MOST+ was designed to be scalable through the integration of multiple modes of web-based support, thus enabling varying levels of direct support by peer moderators and clinicians.

    MOST+ Intervention Components

    Interactive, User-Directed Psychosocial Interventions

    MOST+ adopted a strengths-based approach [42] through which users were guided and prompted to identify (via an interactive card-sort game) and exercise their core personal strengths to foster psychological well-being, including social connectedness and self-efficacy. Each strength included behavioral prompts or do its designed to support young people in applying their core strengths (eg, courage) for specific purposes (eg, dealing with social anxiety).

    Psychosocial interventions in MOST+ took the form of brief web-based comics called steps. Comics have been used in physical health interventions as a nonthreatening, easy-to-understand medium for patient education [72]. We chose to use comics in MOST+ because of their ability to merge persuasive metaphors and character-driven narratives [73], potentially making therapeutic concepts more accessible, engaging, and compelling for young people. Comics were developed by clinical psychologists, professional novelists, and comic artists in partnership with young people (Figure 1). This process included focus groups with young people to identify salient therapeutic themes as well as continual feedback and co-design sessions involving artists, novelists, psychologists, and young people across all phases of comic development (from initial scripting to comic drawing and coloring) [74,75]. MOST+ included 52 discrete, interactive therapy comics designed to address the main concerns for helping young people to access web counseling at eheadspace, including managing immediate distress, low mood, anxiety, social anxiety, relationship issues, and difficulties at school or work [49]. The comics were informed by evidence-based psychological therapies including CBT [76-78], mindfulness [79,80], self-compassion [81], and positive psychology interventions [82]. The comics were designed to be race, age, and gender neutral, with the purpose of maximizing their cultural, sexuality, and gender acceptance. The steps further incorporated do its to help participants exercise therapeutic skills (eg, empathy) in a real-life context (eg, school). Young people were able to store any do it they wanted to complete in the future in a playlist. Finally, the participants had the ability to rate, like, comment on, and share any step or do it with others via the social networking newsfeed.

    Figure 1. Extract of a mindfulness online comic.
    View this figure
    Web-Based Social Networking

    Participants with full access were able to communicate with one another and the peer moderators in the Café and Talk it Out sections to foster social support and problem-solving skills. The Café included a newsfeed where participants and moderators were able to create posts to share thoughts, information, pictures, and videos and respond to other users’ posts by commenting, liking, or reacting to content. The reactions were designed to facilitate social support (eg, I get you and thinking of you). Each participant could personalize their own profile with images and a bio and could visit the wall of fellow users, where their posts and general activities were displayed.

    Talk it Out was designed as a collaborative problem-solving feature informed by the evidence-based problem-solving framework [83,84]. Participants could suggest topics (eg, how to make new friends at a new school in year 11?) and discuss solutions with trained peer moderators and other young people. Peer moderators encouraged young people to define the problem, brainstorm possible solutions, identify the pros and cons, and summarize possible solutions. Previous problems and group solutions were stored in the system, providing an easily accessible solution wiki for all young people.

    Web Counseling

    The web chat was fully integrated within MOST+. Young people using the system could request access to a clinician-delivered web chat between 4 PM and 12 AM. This included real-time web counseling focused on reducing immediate distress, supporting positive self-care, and facilitating referral to additional support where appropriate. Following a web chat session and based on the context of the consultation, MOST+ clinicians suggested specific, relevant content from MOST+ (eg, web-based comic, Talk it Out, or do it) to the young person. These suggestions appeared in the users’ notifications sections as well as in the chat window. Web chats were classified based on the level of risk, with chat requests indicating that suicidal ideation and psychotic symptoms were prioritized.

    Partial Versus Full Access to MOST+

    Participants who consented to the study completed a 15-min web-based survey [56]. Following this survey, all young people were granted partial access to MOST+ consisting of real-time, clinician-delivered web chat as well as user-directed psychosocial interventions. Full access to MOST+, which additionally included peer-to-peer web-based social networking, was granted based on a three-tiered screening process designed to ascertain the safety and appropriateness of this component for each young person (Multimedia Appendix 1).

    Duration of Access to MOST+

    Irrespective of the level of access, all participants were enrolled in the MOST+ intervention for 1 week, with the option to extend their enrollment on a weekly basis over the duration of the intervention period (ie, a minimum of 1 week to a maximum of 9 weeks). The participants were shown a count-down watch depicting the time left before their account deactivation. Following account deactivation, all information and activity pertaining to the participant’s account (eg, profile, posts) were hidden from the MOST+ system. However, the participants were able to reactivate an expired account at any time during the study intervention period. Multimedia Appendix 2 provides examples of the participants’ enrollment timelines, with each colored line representing a different potential timeline over the duration of the pilot. This feature was implemented to protect the privacy of the large group of one-time users of eheadspace services [49] while ensuring a lively and safe online social network for regular users. Specifically, based on the patterns of usage of eheadspace, we anticipated that a significant proportion of users would only use MOST+ for a short period. By automatically hiding their activity following their short-term use of MOST+, we aimed to protect their privacy in the long term, while maintaining the currency and dynamism of the social network and facilitating the safety management of the social network at scale.

    Moderation

    MOST+ incorporated clinician as well as peer moderation. Clinical moderation primarily focused on ensuring the safety of the social network. Specifically, MOST+ clinicians monitored new contributions to the network for indicators of clinical risk. The social network was moderated by an on-duty MOST+ clinician daily. Safety checks (ie, monitoring any indications of risk on the social network) were undertaken a minimum of 2 times per week day and once daily on weekends and public holidays (see Safety Protocol details below). The clinicians were regularly supervised throughout the trial by their usual eheadspace shift supervisors.

    The Cafe was led and moderated by trained young people with the experience of having lived with mental illness (Peer workers). Peer workers were peer moderators who facilitated social learning using MOST+ in desired ways (eg, self-disclosing, using therapy content to deal with difficulties) [85]. Drawing on the growing evidence for peer support having a positive impact on the levels of hope, empowerment, and quality of life [86], peer workers also provided general guidance and peer-to-peer support. In addition, peer workers guided the problem-solving discussions in Talk it Out and posted links to relevant therapeutic resources and tips. Finally, peer workers seeded discussion threads and icebreakers to enable useful, enjoyable conversations and facilitate meaningful relationships. Peer moderators were supervised weekly by members of the research team (clinical psychologists and peer support coordinators).

    Safety Protocol

    The safety protocol comprised 3 levels of security: (1) system and privacy protection, (2) web safety, and (3) clinical safety. MOST+ had built-in security and data protection to prevent unauthorized access to the platform, which has been described elsewhere [56]. These measures conform to industry best practices as defined by the Open Web Application Security Project. Privacy and web safety were managed in accordance with the Australian Communications and Media Authority. Specifically, the participants were informed of, and were required to accept, the terms of use of MOST+, which included clauses on protecting their privacy and that of others as well as guidelines on proscribed behavior (ie, disrespectful behavior or offensive comments).

    The MOST+ clinical safety protocol included manual and automated procedures. First, information related to clinical risk (posts or messages) was screened by clinical moderators twice each weekday and daily on weekends and public holidays. Second, MOST+ incorporated an automatic alert system that monitored self-harm–related terms posted on the social feed. Any detected increased risk or inappropriate use activated the safety protocol (Multimedia Appendix 3). Finally, a report function enabled the participants to alert the moderators to inappropriate use of the system (eg, discriminatory comments posted on the social network).

    Participants’ Data Management

    The participants were able to control the extent to which they could be identified by other users within the social network, including whether they used their first name or a nickname and whether their profile picture included a photo. As noted above, following account deactivation, the participants’ accounts and activities were hidden from MOST+. The participants could also choose to switch off their profiles to hide all past posts and comments and anonymize their contributions to Talk it Out. The participants were informed that any records of user activity hidden from the social network were retained by the researchers for the purpose of analysis. Specifically, all user-generated data were encrypted and retained in the MOST+ database throughout the trial. Upon completion of the trial, MOST+ was decommissioned and the database was exported and stored in a deidentified format on an Orygen server for research purposes.

    Outcome Measures

    The primary outcome variables were intervention feasibility, acceptability, and safety. All outcomes were assessed at baseline and at follow-up. Baseline assessments were conducted on the web as part of the onboarding process. Follow-up assessment occurred approximately 4 days after the initial account deactivation (ie, 4 days after a participant opted not to renew their account for an additional week). For those participants who maintained active enrollment across the intervention period, follow-up occurred as soon as possible following the conclusion of the pilot. The participants received a short message service notification indicating that their web follow-up survey was due and that they were able to complete survey items either via the web or telephone.

    A self-report user feedback questionnaire was developed based on the user experience approach [87] assessing the following themes: (1) acceptability, (2) helpfulness, and (3) safety. Acceptability was determined against the following a priori indicators: (1) participants provided ratings of the MOST+ platform averaging above three out of five across feedback questions regarding ease of use, relevancy, helpfulness, and overall experience; (2) at least 60% (56/93) of the participants reported that the MOST+ intervention provided relevant and helpful support; and (3) at least 80% (74/93) of the participants would recommend MOST+ to other young people experiencing similar difficulties (Table 2). In addition, the MOST+ intervention was considered safe if (1) at least 90% (84/93) of the participants reported the web-based intervention to be safe, (2) none of the participants experienced a serious adverse event as a result of their engagement with the system, and (3) there were no unlawful entries into the MOST+ system detected by study programmers during the 8-week pilot.

    Secondary outcome measures included self-report measures of psychological distress, well-being, depression, stress, social support, loneliness, basic psychological needs (self-competence, relatedness, and autonomy), strengths usage, and mindfulness skills (Table 3).

    Table 2. Acceptability, safety, and perceived helpfulness ratings using Enhanced Moderated Online Social Therapy (N=93).
    View this table
    Table 3. Overview of secondary outcomes and measures used.
    View this table

    Analyses

    The patterns of intervention use were tracked in real time. Aggregated data from the user feedback questionnaire were compared with the a priori acceptability and safety criteria to determine the success of the pilot. Paired samples t tests were conducted and within-group effect sizes (Cohen d) were reported for changes in the baseline and posttest study measures. To estimate Cohen d, Morris and DeShon’s [88] equation was applied to correct for dependence among the means in within-group designs. Parametric and nonparametric correlations were performed as appropriate to explore the associations between the usage of MOST+, acceptability ratings, and changes in secondary outcome measures.


    Results

    Feasibility, Acceptability, and Safety

    A total of 93 of the 157 participants recruited for the study were contactable and assessed at follow-up. There were no statistically significant differences in any baseline demographic or clinical variables between those who completed the follow-up assessment and those who were lost to follow-up. All a priori indicators of acceptability were met (Table 2). The participants provided positive ratings of their experience using MOST+, with mean scores of 3.5 or more (out of 5) on each of the core domains of ease of use (mean 3.7, SD 1.1), relevancy (mean 3.9, SD 1.0), helpfulness (mean 3.5, SD 0.9), and overall experience (mean 3.9, SD 0.8). In addition, 98% (91/93) of participants reported a positive experience using MOST+, 86% (80/93) considered it easy to use, 88% (88/93) reported that MOST+ was relevant to their needs, 86% (80/93) considered it helpful, 82% (76/93) reported that using MOST+ helped them feel better, 86% (70/93) felt more socially connected using it, and 92% (86/93) said that they would recommend it to other young people experiencing similar difficulties. Moreover, 46% (72/157) of the participants had full access to MOST+ and 53% (83/157) had partial access (ie, excluded from the web-based social networking). The reasons for partial access included high clinical risk either detected by the system (automatically triggering a web chat; 18%: 28/157 (of all participants) or based on the clinician’s assessment (4%: 6/157) and not being able to contact eligible participants to verify age (31%: 49/157). Participants with full access to MOST+ reported a significantly more positive overall experience (mean 4.1, SD 0.7) compared with those with partial access (mean 3.7, SD 0.8; two-tailed t(91)=−2.89; P<.001). The follow-up retention rate was also significantly higher in participants with full access (57/73, 78%) than in those with partial access (43%; X2155=20.1; P<.001). There were no other significant differences in any demographic or outcome variable at baseline or acceptability ratings at follow-up between participants with full versus partial access.

    A priori set safety criteria were also met. Specifically, no adverse events, inappropriate use, reports by participants, or unlawful entries pertaining to MOST+ were detected during the study. A total of 97% (90/93) of the participants reported feeling safe using MOST+. Moreover, all clinical measures showed a trend toward improved clinical status at follow-up (Table 1). There were no significant differences in safety indicators between those with full versus partial access.

    Regarding the overall use of MOST+, there were a total of 1058 log-ins during the 9-week study, with 45.2% (71/157) logging in once, 14% (22/157) logging in twice, and 40.8% (64/157) logging in 3 or more times (Table 4). A total of 585 steps and 244 do its were completed during the study. All indicators of system usage were significantly higher in those with full access to MOST+ compared with those with partial access (Table 5). In terms of the duration of access to MOST+, 66.2% (104/157) of participants did not extend their initial default period of 7 days of enrollment, 14% (22/157) requested one extension, and 19.7% (31/157) requested 2 or more extensions. Finally, 41% (64/157) of participants did not request any chats with eheadspace clinicians, 34% (53/157) requested 1 chat, and 24% (38/157) requested 3 or more chats (including automatic web chats triggered by screening items indicating possible acute risk on initial registration into MOST+; Multimedia Appendix 1).

    Table 4. Log-ins and individual usage of the main components of Enhanced Moderated Online Social Therapy (N=157) during the pilot study.
    View this table
    Table 5. Comparison of log-ins and individual use of the main components between participant groups with full access (n=73) and participant group with partial access (n=84).
    View this table

    Secondary Outcome Variables

    There were statistically significant improvements between baseline and follow-up assessments, with a small to medium size, in psychological distress (d=−0.39; P<.001), perceived stress (d=−0.44; P<.001), psychological well-being (d=0.51; P<.001), depression (d=−0.29; P=.008), loneliness (d=−0.23; P=.04), social support (d=0.30; P<.001), autonomy (d=0.36; P=.001), and self-competence (d=0.30; P<.001; Table 3). Moreover, the proportion of participants with K10 and PHQ-9 scores indicative of severe mental health disorder (K1030) or moderately severe depression (PHQ-915) was significantly lower at follow-up (72% for K10; 53% for PHQ-9) compared with baseline (82% for K10; 58% for PHQ-9; X291=18.8; P<.001 for K10; X291=19.7; P<.001 for PHQ-9).

    For those with full access, a secondary analysis revealed that there were significant improvements in psychological distress (d=−0.38; P<.001), perceived stress (d=−0.37; P=.01), psychological well-being (d=0.38; P<.001), loneliness (d=−0.33; P=.02), social support (d=0.25; P=.05), and autonomy (d=0.50; P<.001). Similarly, for those with partial access to MOST+, there were significant improvements in psychological distress (d=−0.40; P=.03), perceived stress (d=−0.55; P<.001), psychological well-being (d=0.72; P<.001), depression (d=−0.40; P=.03), social support (d=0.39; P=.03), and self-competence (d=0.42; P<.001).

    Exploratory Correlations Between System Use, Acceptability Ratings, and Outcome Variables

    Given the significantly lower system usage and overall retention rate in participants with partial access compared with those with full access, we reported exploratory correlations between system usage, acceptability ratings, and secondary outcome variables for participants who had full access to MOST+. In terms of system usage and acceptability ratings, there were significant correlations between (1) participants reporting that MOST+ helped them feel better and the number of web-based messages between clinicians and young people (Spearman rho, rs=0.53; P<.01) and the number of weeks logging in (rs=0.42; P<.01) and (2) participants reporting feeling more socially connected and the number of comments posted on the newsfeed (rs=0.42; P<.01) as well as the number of contributions to Talk it Out (rs=0.32; P=.01). With respect to system usage and secondary outcome variables, there were statistically significant correlations between increased relatedness and the number of log-ins (rs=0.28; P=.03), number of steps completed (rs=0.37; P<.01), as well as the combined number of do its and steps completed (rs=0.40; P<.01). Increased social support also correlated positively with the number of completed steps (rs=0.30; P=.02) and combined number of do its and steps. Moreover, increased strengths usage correlated positively with the number of completed do its (rs=0.27; P=.03). Finally, there was a nonsignificant correlation in the expected direction between lower loneliness and the number of log-ins (rs=−0.25; P=.06).


    Discussion

    To the best of our knowledge, this is the first study to develop and test a multimodal nationwide web-based mental health service for young people experiencing mental ill-health. As such, MOST+ was designed to be an all-in-one digital mental health app merging engaging, evidence-based therapy modules with expert clinician guidance, peer support, social networking, and real-time clinical support. Baseline clinical measures indicated that the majority of participants using MOST+ had moderate (52/157, 33%) to severe mental health conditions (96/157, 61%) and moderate to severe depressive symptoms (130/157, 83%). The results of this study showed that MOST+ was feasible, acceptable, and safe, with all acceptability and safety indicators exceeding the a priori established criteria. The high level of overall satisfaction and perceived helpfulness provided strong support for the relevance of the intervention content and features for help-seeking young people experiencing significant mental ill-health.

    Secondary outcome variables showed significant improvements, with small to medium effect sizes, in 8 of the 11 outcomes assessed. These included psychological distress, perceived stress, psychological well-being, depression, loneliness, social support, autonomy, and self-competence. Similarly, the proportions of participants with severe mental health disorders and moderate to severe depression (as indicated by the K-10 and PHQ-9) were significantly lower at follow-up. Although the uncontrolled design of this study did not allow any causal inferences, it was worth noting that there were a number of significant correlations in the expected direction between several indicators of system usage (ie, number of log-ins, number of steps completed, number of do its completed, and number of posts made on the social network) and both perceived helpfulness (ie, participants reporting that MOST+ made them feel better and more socially connected) as well as secondary outcome variables (ie, relatedness and loneliness, strengths usage, and social support). These initial findings provide proof of concept for MOST+ and lend preliminary support to the potential therapeutic effects of the intervention.

    MOST+ was designed as a scalable and efficient online youth mental health service integrating multiple modes of digital therapy, available 24/7, thus catering to individual needs and preferences of young people. The combination of treatment modalities integrated by MOST+ was reported in previous research [38,47,48,89-91]. Specifically, recent studies have found that web e chat can be as effective as face-to-face interventions and even out-perform face-to-face treatment, possibly via increased focus on essential treatment goals [47,48]. Writing in web chats can also have significant advantages, such as enabling users to re-read and reflect on the therapist’s responses and their own emotions and promote a sense of control of the pace, content, and depth of disclosure [48]. Moreover, online chat integrated with a web-based intervention was shown to increase clinical effects compared with web-based support alone [89]. Finally, online peer support showed promise in improving mental health outcomes [38,90], potentially improving intervention adherence and satisfaction [45,92], although some studies found no additional therapeutic effect of peer support compared with traditional online interventions [91]. The patterns of use of MOST+, with different young people interacting with different features (ie, steps, web chat with clinicians, and web-based social network) over different timeframes (ie, one-time usage vs multiple extensions) coupled with the finding that 88% (81/93) of the participants reported that the intervention was relevant to their needs, provide support to this approach. Furthermore, the fact that 41% (64/157) of young people did not request a web chat while using MOST+ indicated that this model could increase the efficiency and scalability of traditional web chat services by reducing their reliance on real-time clinician-delivered web chat. Although promising, the cost-effectiveness of MOST+ against traditional web chat services needs to be established via controlled studies.

    With the purpose of ensuring the safety of a (potentially) population-level social network, participants could be granted either full or partial access (excluded access to web-based social networking) to MOST+. This provided an opportunity to examine the differences in satisfaction levels, perceived safety, usage, and secondary outcomes in relation to the level of access to the system. Interestingly, participants with access to the social network reported significantly higher levels of overall satisfaction, showed higher levels of usage of MOST+, and were more likely to be interviewed at follow-up (60/73, 78%) compared with their counterparts in the partial access group (36/84, 43%). Moreover, although loneliness and autonomy improved significantly in the group with full access, these domains remained unchanged in the partial access group. Taken together, these findings suggested that limiting access to the social network may have thwarted the sense of autonomy and the motivation to use the system and remained in the study in those with partial access. Conversely, having full access to the system and social network may lead to increased autonomy and reduced loneliness, irrespective of whether young people posted or not. It must be noted that there were no significant differences in any baseline variables (including clinical severity and basic psychological needs) between those with partial access and those with full access. These findings were in keeping with the self-determination theory, which posited that environments that addressed the basic psychological needs of autonomy (ie, sense that one’s own behavior is freely chosen and of one’s own volition), relatedness (ie, feelings of safety, belonging, and connectedness in their social interactions), and self-competence enhanced intrinsic motivation [93,94] as well as engagement in web-based interventions [95]. Alternatively, it could have been that participants with partial access were more difficult to contact because they were less motivated to use the system and participate in the study in the first place. Taken as a whole, these findings highlighted that, when designing web-based social media–based interventions, safety considerations should be weighed against potential negative effects on engagement or lack of positive effects on social outcomes. For example, future iterations of MOST+ could include an opt-in feature whereby young people decide whether they want to participate in the social network, thus preserving their sense of autonomy. In addition, less burdensome procedures could be introduced to verify age, which accounted for 57% (48/84) of all participants with partial access (eg, uploading an ID card picture via mobile phone as part of the onboarding process or linking their MOST+ account to the existing national web health records to automatically verify age). For those at increased clinical risk, additional safety measures (eg, enhanced automatic and manual monitoring of posts; personalized, detailed information on the purpose and terms of use of the social network) could be implemented to ensure clinical safety while including them in the social network. We successfully implemented this approach in a recent study of a social media–based intervention for young people with elevated suicidal ideation [96].

    Web-based social media interventions provide a unique opportunity to address the pervasive rates of social isolation and lack of social support among young people with mental ill-health. For example, young people with psychosis report an average of three lonely days per week [97]. Come adulthood, 75%-94% will experience significant loneliness [98,99]. Unfortunately, recent studies have shown that frequent use of social media among young people is linked to lower self-esteem and increased anxiety, depression, and psychological distress [100,101], with young people with lower well-being being more vulnerable to experience adverse effects when using social media [102]. Against this backdrop, it is essential to develop evidence-based social media–enabled interventions that promote social support while ensuring safety and diminishing harmful consequences. A number of studies from our research lab [42-44,55] and others [39] have shown that carefully designed moderated web-based social media interventions can be safely implemented and are not associated with harmful effects. This study adds to this growing body of evidence by demonstrating that web-based social media interventions can be safely deployed to help-seeking young people via a national web counseling service. That said, the optimal size, functioning, and operations of social media–based interventions remain uncertain. For example, what level of engagement or participation is needed for participants to benefit from these interventions? What is the optimal balance between messages of distress and requests for help (which may lead to contagion [103]) versus messages of hope and positivity (which may alienate some participants [104])? Can web-based social networks provide a safe and transitional training environment that leads to real-world improvements? These and other questions will need to be examined in future research using mixed methods, including qualitative studies as well as novel methodologies and analytic techniques such as machine learning and natural language processing.

    This study has several limitations. First, the uncontrolled design precluded any causal inferences about the efficacy of MOST+. Second, given that the study was implemented nationally and all assessments were conducted remotely, there was a 40% attrition rate at follow-up, which may have positively biased the results (ie, young people who felt more positively about the intervention may be more likely to be assessed at follow-up). That being said, the reported attrition rate is among the lowest reported by studies evaluating web-based interventions of the equivalent duration via remote assessments (35%-74%) [105-107], and there were no differences in any baseline demographic or clinical variables between those who completed the follow-up assessment and those who did not. Moreover, it is worth noting that MOST+ was not designed to promote sustained engagement, and it did not implement any strategies to foster engagement over time. On the basis of the purpose and function of the eheadspace web counseling service, MOST+ was intended to provide immediate short-term support to help-seeking young people and, where appropriate, redirect young people with long-term needs to youth mental health services. Third, multiple correlations were estimated, which is likely to increase the number of type I errors. Given the exploratory nature of these analyses, we did not adjust for multiple comparisons. Thus, these findings should be considered to be exploratory and interpreted with caution. Fourth, the short-term duration of the study precluded the examination of long-term outcomes. However, this design was considered appropriate, given the purpose of the intervention (ie, addressing immediate psychological distress) and the typically short-term engagement of young people with eheadspace. Finally, the multimodal nature of MOST+ precludes the examination of the effectiveness of the specific elements of MOST+, leading to improved outcomes. Although these research questions were outside the scope of this study, future research adopting novel approaches (eg, rapid A/B testing) will need to determine the optimal combination of features, therapeutic content, and level of usage to improve technological markers (eg, penetration rate, satisfaction, and perceived helpfulness) as well as clinical (eg, symptoms) and social (eg, loneliness, social support) outcomes. Moreover, further research should determine whether MOST+ is effective in fostering help-seeking in, and addressing the needs of, young people with lower socio-economic status or young people at the risk of social exclusion.

    Conclusions and Future Research

    The results of this pilot investigation demonstrated that MOST+ is a highly promising and relevant web-based clinical service for young people with clinically significant mental ill-health as it yielded high satisfaction, safety, and perceived helpfulness as well as encouraging improvements in a wide range of clinical and social outcomes. These initial findings provide proof of concept for MOST+ and lend support to the multimodal, integrated approach of the intervention.

    The effectiveness and cost-effectiveness of MOST+ will need to be established via controlled evaluations addressing the limitations of this study. For example, MOST+ could be implemented as a national service and evaluated through hybrid trial designs that blend components of clinical effectiveness and implementation research [108]. Alternatively, the national deployment of MOST+ would enable fast, well-powered, efficient randomized controlled trials evaluating the effectiveness of the different components of the intervention (eg, dismantling trials) as well as successive iterations of the service. The results and innovations of these trials could be rapidly assimilated into the mainstream service, thereby breaking the current divide and long-term delays between research and clinical implementation [109].

    The results from this study indicate that MOST+ is a scalable web-based mental health service that enhances the capacity of traditional web counseling services. Future iterations of MOST+ will incorporate artificial intelligence (AI) and machine learning technologies to further enhance the efficiency of the service (eg, via triaging human support as required) as well as the personalization of the intervention [110]. This could also include chatbots harnessing natural language processing and AI to support participants in finding relevant content within the system, delivering basic therapeutic counseling, and initiating human involvement as needed. Finally, optional built-in video conferencing capabilities may enable faster therapeutic sessions while being able to assess nonverbal cues when necessary.

    Finally, in addition to providing mental health support to young people who are not able to access face-to-face care, MOST+ could be integrated with the growing international network of youth mental health services to address wait-list issues, provide continuity of care in between therapy sessions, and offer relapse prevention support after initial treatment response. Meanwhile, MOST+, in its current form, stands to deliver an accessible and scalable web-based mental health service, providing multiple and integrated modalities of web-based support, to cater to the needs of an increasingly growing number of young people with mental ill-health.

    Acknowledgments

    The authors wish to thank the young people who generously participated in this study. In addition, the authors wish to thank the eheadspace senior management team and clinicians for supporting this study. This work was supported by a grant from the Young and Well Cooperative Research Centre funded by the National Health and Medical Research Council (NHMRC). MA was supported via a Career Development Fellowship (APP1082934) and an Investigator Grant (APP1177235) by NHMRC. The sponsors did not participate in the design or conduct of this study; in the collection, management, analysis, or interpretation of data; in the writing of the manuscript; or in the preparation, review, approval, or decision to submit this manuscript for publication.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Study procedure. MOST+: Enhanced Moderated Online Social Therapy.

    PNG File , 576 KB

    Multimedia Appendix 2

    Example participant timelines through the Enhanced Moderated Online Social Therapy (MOST+) intervention.

    PNG File , 342 KB

    Multimedia Appendix 3

    Enhanced Moderated Online Social Therapy (MOST+) intervention safety algorithm.

    PNG File , 288 KB

    References

    1. Saxena S, Funk MK, Chisholm D. Comprehensive mental health action plan 2013-2020. East Mediterr Health J 2015 Sep 28;21(7):461-463 [FREE Full text] [Medline]
    2. Walker ER, McGee RE, Druss BG. Mortality in mental disorders and global disease burden implications: a systematic review and meta-analysis. JAMA Psychiatry 2015 Apr;72(4):334-341 [FREE Full text] [CrossRef] [Medline]
    3. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch Gen Psychiatry 2005 Jun;62(6):593-602. [CrossRef] [Medline]
    4. Alvarez-Jimenez M, Priede A, Hetrick S, Bendall S, Killackey E, Parker A, et al. Risk factors for relapse following treatment for first episode psychosis: a systematic review and meta-analysis of longitudinal studies. Schizophr Res 2012 Aug;139(1-3):116-128. [CrossRef] [Medline]
    5. Melvin GA, Dudley AL, Gordon MS, Ford S, Taffe J, Tonge BJ. What happens to depressed adolescents? A follow-up study into early adulthood. J Affect Disord 2013 Oct;151(1):298-305. [CrossRef] [Medline]
    6. Patel V, Flisher AJ, Hetrick S, McGorry P. Mental health of young people: a global public-health challenge. Lancet 2007 Apr 14;369(9569):1302-1313. [CrossRef] [Medline]
    7. Naicker K, Galambos NL, Zeng Y, Senthilselvan A, Colman I. Social, demographic, and health outcomes in the 10 years following adolescent depression. J Adolesc Health 2013 May;52(5):533-538. [CrossRef] [Medline]
    8. Draine J, Salzer MS, Culhane DP, Hadley TR. Role of social disadvantage in crime, joblessness, and homelessness among persons with serious mental illness. Psychiatr Serv 2002 May;53(5):565-573. [CrossRef] [Medline]
    9. Gibb SJ, Fergusson DM, Horwood LJ. Burden of psychiatric disorder in young adulthood and life outcomes at age 30. Br J Psychiatry 2010 Aug;197(2):122-127. [CrossRef] [Medline]
    10. Slade J, Teesson W, Burgess P. The Mental Health of Australians 2: Report on the 2007 National Survey of Mental Health and Wellbeing. Australian Government Department of Health. 2009.   URL: https://www1.health.gov.au/internet/main/publishing.nsf/Content/mental-pubs-m-mhaust2 [accessed 2020-07-21]
    11. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry 2010 Dec 30;10:113 [FREE Full text] [CrossRef] [Medline]
    12. Thornicroft G. Stigma and discrimination limit access to mental health care. Epidemiol Psichiatr Soc 2008;17(1):14-19. [CrossRef] [Medline]
    13. Salaheddin K, Mason B. Identifying barriers to mental health help-seeking among young adults in the UK: a cross-sectional survey. Br J Gen Pract 2016 Oct;66(651):e686-e692 [FREE Full text] [CrossRef] [Medline]
    14. Jorm AF, Wright A, Morgan AJ. Where to seek help for a mental disorder? National survey of the beliefs of Australian youth and their parents. Med J Aust 2007 Nov 19;187(10):556-560. [Medline]
    15. Anderson M, Jiang J. Teens, Social Media & Technology 2018. Pew Research Center. 2018.   URL: https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/ [accessed 2020-07-21]
    16. Rideout V, Robb M. Social Media, Social Life: Teens Reveal Their Experiences, 2018. Common Sense Media: Age-Based Media Reviews for Families. 2018.   URL: https://www.commonsensemedia.org/research/social-media-social-life-2018 [accessed 2020-07-21]
    17. Birnbaum ML, Rizvi AF, Confino J, Correll CU, Kane JM. Role of social media and the internet in pathways to care for adolescents and young adults with psychotic disorders and non-psychotic mood disorders. Early Interv Psychiatry 2017 Aug;11(4):290-295 [FREE Full text] [CrossRef] [Medline]
    18. Lawlor A, Kirakowski J. Online support groups for mental health: a space for challenging self-stigma or a means of social avoidance? Comput Hum Behav 2014 Mar;32:152-161. [CrossRef]
    19. Pretorius C, Chambers D, Coyle D. Young people's online help-seeking and mental health difficulties: systematic narrative review. J Med Internet Res 2019 Nov 19;21(11):e13873 [FREE Full text] [CrossRef] [Medline]
    20. Best P, Gil-Rodriguez E, Manktelow R, Taylor BJ. Seeking help from everyone and no-one: conceptualizing the online help-seeking process among adolescent males. Qual Health Res 2016 Jul;26(8):1067-1077. [CrossRef] [Medline]
    21. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions — a systematic review. Internet Interv 2014 Oct;1(4):205-215. [CrossRef]
    22. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    23. Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered treatment for generalized anxiety disorder: a systematic review and meta-analysis. Internet Interv 2015 Sep;2(3):272-282. [CrossRef]
    24. Mohr DC, Weingardt KR, Reddy M, Schueller SM. Three problems with current digital mental health research . . . and three things we can do about them. Psychiatr Serv 2017 May 1;68(5):427-429 [FREE Full text] [CrossRef] [Medline]
    25. Mohr DC, Riper H, Schueller SM. A solution-focused research approach to achieve an implementable revolution in digital mental health. JAMA Psychiatry 2018 Feb 1;75(2):113-114. [CrossRef] [Medline]
    26. Aitken M, Clancy B, Nass D. The Growing Value of Digital Health. IQVIA. 2017.   URL: https://www.iqvia.com/insights/the-iqvia-institute/reports/the-growing-value-of-digital-health [accessed 2020-07-21]
    27. Lau N, O'Daffer A, Colt S, Yi-Frazier JP, Palermo TM, McCauley E, et al. Android and iphone mobile apps for psychosocial wellness and stress management: systematic search in app stores and literature review. JMIR Mhealth Uhealth 2020 May 22;8(5):e17798 [FREE Full text] [CrossRef] [Medline]
    28. Leigh S, Flatt S. App-based psychological interventions: friend or foe? Evid Based Ment Health 2015 Nov;18(4):97-99. [CrossRef] [Medline]
    29. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res 2015 Aug 17;17(8):e198 [FREE Full text] [CrossRef] [Medline]
    30. Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [CrossRef] [Medline]
    31. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
    32. Weisel KK, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, Ebert DD. Standalone smartphone apps for mental health-a systematic review and meta-analysis. NPJ Digit Med 2019;2:118 [FREE Full text] [CrossRef] [Medline]
    33. Buntrock C, Ebert DD, Lehr D, Smit F, Riper H, Berking M, et al. Effect of a web-based guided self-help intervention for prevention of major depression in adults with subthreshold depression: a randomized clinical trial. J Am Med Assoc 2016 May 3;315(17):1854-1863. [CrossRef] [Medline]
    34. Kampmann IL, Emmelkamp PM, Morina N. Meta-analysis of technology-assisted interventions for social anxiety disorder. J Anxiety Disord 2016 Aug;42:71-84. [CrossRef] [Medline]
    35. Kuester A, Niemeyer H, Knaevelsrud C. Internet-based interventions for posttraumatic stress: a meta-analysis of randomized controlled trials. Clin Psychol Rev 2016 Feb;43:1-16. [CrossRef] [Medline]
    36. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [CrossRef] [Medline]
    37. Mohr DC, Lyon AR, Lattie EG, Reddy M, Schueller SM. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res 2017 May 10;19(5):e153 [FREE Full text] [CrossRef] [Medline]
    38. Rice SM, Goodall J, Hetrick SE, Parker AG, Gilbertson T, Amminger GP, et al. Online and social networking interventions for the treatment of depression in young people: a systematic review. J Med Internet Res 2014 Sep 16;16(9):e206 [FREE Full text] [CrossRef] [Medline]
    39. Easton K, Diggle J, Ruethi-Davis M, Holmes M, Byron-Parker D, Nuttall J, et al. Qualitative exploration of the potential for adverse events when using an online peer support network for mental health: cross-sectional survey. JMIR Ment Health 2017 Oct 30;4(4):e49 [FREE Full text] [CrossRef] [Medline]
    40. Naslund JA, Aschbrenner KA, Marsch LA, Bartels SJ. The future of mental health care: peer-to-peer support and social media. Epidemiol Psychiatr Sci 2016 Apr;25(2):113-122 [FREE Full text] [CrossRef] [Medline]
    41. Ridout B, Campbell A. The use of social networking sites in mental health interventions for young people: systematic review. J Med Internet Res 2018 Dec 18;20(12):e12244 [FREE Full text] [CrossRef] [Medline]
    42. Alvarez-Jimenez M, Bendall S, Lederman R, Wadley G, Chinnery G, Vargas S, et al. On the HORYZON: moderated online social therapy for long-term recovery in first episode psychosis. Schizophr Res 2013 Jan;143(1):143-149. [CrossRef] [Medline]
    43. Alvarez-Jimenez M, Gleeson J, Bendall S, Penn D, Yung A, Ryan R, et al. Enhancing social functioning in young people at ultra high risk (UHR) for psychosis: a pilot study of a novel strengths and mindfulness-based online social therapy. Schizophr Res 2018 Dec;202:369-377. [CrossRef] [Medline]
    44. Rice S, Gleeson J, Davey C, Hetrick S, Parker A, Lederman R, et al. Moderated online social therapy for depression relapse prevention in young people: pilot study of a 'next generation' online intervention. Early Interv Psychiatry 2018 Aug;12(4):613-625. [CrossRef] [Medline]
    45. Lederman R, Wadley G, Gleeson J, Bendall S, Álvarez-Jiménez M. Moderated online social therapy. ACM Trans Comput-Hum Interact 2014 Feb;21(1):1-26. [CrossRef]
    46. Dowling M, Rickwood D. Online counseling and therapy for mental health problems: a systematic review of individual synchronous interventions using chat. J Tech Hum Serv 2013 Jan;31(1):1-21. [CrossRef]
    47. Gieselmann A, Pietrowsky R. The effects of brief chat-based and face-to-face psychotherapy for insomnia: a randomized waiting list controlled trial. Sleep Med 2019 Sep;61:63-72. [CrossRef] [Medline]
    48. Richards D, Viganó N. Online counseling: a narrative and critical review of the literature. J Clin Psychol 2013 Sep;69(9):994-1011. [CrossRef] [Medline]
    49. Rickwood D, Webb M, Kennedy V, Telford N. Who are the young people choosing web-based mental health support? Findings from the implementation of Australia's national web-based youth mental health service, eHeadSpace. JMIR Ment Health 2016 Aug 25;3(3):e40 [FREE Full text] [CrossRef] [Medline]
    50. Alvarez-Jimenez M, Bendall S, Koval P, Rice S, Cagliarini D, Valentine L, et al. HORYZONS trial: protocol for a randomised controlled trial of a moderated online social therapy to maintain treatment effects from first-episode psychosis services. BMJ Open 2019 Feb 19;9(2):e024104 [FREE Full text] [CrossRef] [Medline]
    51. Bailey E, Alvarez-Jimenez M, Robinson J, D'Alfonso S, Nedeljkovic M, Davey CG, et al. An enhanced social networking intervention for young people with active suicidal ideation: safety, feasibility and acceptability outcomes. Int J Environ Res Public Health 2020 Apr 3;17(7):2435 [FREE Full text] [CrossRef] [Medline]
    52. Rice S, O'Bree B, Wilson M, McEnery C, Lim MH, Hamilton M, et al. Leveraging the social network for treatment of social anxiety: pilot study of a youth-specific digital intervention with a focus on engagement of young men. Internet Interv 2020 Apr;20:100323 [FREE Full text] [CrossRef] [Medline]
    53. McEnery C, Lim MH, Knowles A, Rice S, Gleeson J, Howell S, et al. Social anxiety in young people with first-episode psychosis: pilot study of the EMBRACE moderated online social intervention. Early Interv Psychiatry 2019 Dec 30:- epub ahead of print. [CrossRef] [Medline]
    54. Gleeson J, Lederman R, Herrman H, Koval P, Eleftheriadis D, Bendall S, et al. Moderated online social therapy for carers of young people recovering from first-episode psychosis: study protocol for a randomised controlled trial. Trials 2017 Jan 17;18(1):27 [FREE Full text] [CrossRef] [Medline]
    55. Gleeson J, Lederman R, Koval P, Wadley G, Bendall S, Cotton S, et al. Moderated online social therapy: a model for reducing stress in carers of young people diagnosed with mental health disorders. Front Psychol 2017;8:485 [FREE Full text] [CrossRef] [Medline]
    56. Rice S, Gleeson J, Leicester S, Bendall S, D'Alfonso S, Gilbertson T, et al. Implementation of the enhanced moderated online social therapy (MOST+) model within a national youth e-mental health service (eHeadSpace): protocol for a single group pilot study for help-seeking young people. JMIR Res Protoc 2018 Feb 22;7(2):e48 [FREE Full text] [CrossRef] [Medline]
    57. Mueser K, Drake RE. How does a practice become evidenced-based? In: Drake RE, Merrens MR, Lynde DW, editors. Evidence-Based Mental Health Practice: A Textbook. Manhattan, New York: Norton Professional Books; 2005.
    58. Andrews G, Slade T. Interpreting scores on the Kessler psychological distress scale (K10). Aust N Z J Public Health 2001 Dec;25(6):494-497. [CrossRef] [Medline]
    59. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatric Annals 2002 Sep 1;32(9):509-515 [FREE Full text] [CrossRef]
    60. Chan SM, Fung TC. Reliability and validity of K10 and K6 in screening depressive symptoms in Hong Kong adolescents. Vulnerable Child Youth Stud 2013 Nov 25;9(1):75-85. [CrossRef]
    61. Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes 2007 Nov 27;5:63 [FREE Full text] [CrossRef] [Medline]
    62. Lee E. Review of the psychometric evidence of the perceived stress scale. Asian Nurs Res (Korean Soc Nurs Sci) 2012 Dec;6(4):121-127 [FREE Full text] [CrossRef] [Medline]
    63. Beard C, Hsu K, Rifkin L, Busch A, Björgvinsson T. Validation of the PHQ-9 in a psychiatric sample. J Affect Disord 2016 Mar 15;193:267-273. [CrossRef] [Medline]
    64. Arroll B, Goodyear-Smith F, Crengle S, Gunn J, Kerse N, Fishman T, et al. Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Ann Fam Med 2010;8(4):348-353 [FREE Full text] [CrossRef] [Medline]
    65. Russell DW. UCLA loneliness scale (version 3): reliability, validity, and factor structure. J Pers Assess 1996 Feb;66(1):20-40. [CrossRef] [Medline]
    66. Deci EL, Ryan RM. The 'what' and 'why' of goal pursuits: human needs and the self-determination of behavior. Psychol Inq 2000 Oct;11(4):227-268. [CrossRef]
    67. Gagné M. The role of autonomy support and autonomy orientation in prosocial behavior engagement. Motiv Emot 2003;27(3):199-223. [CrossRef]
    68. Johnston MM, Finney SJ. Measuring basic needs satisfaction: evaluating previous research and conducting new psychometric evaluations of the basic needs satisfaction in general scale. Contemp Educ Psychol 2010 Oct;35(4):280-296. [CrossRef]
    69. Hawthorne G. Perceived social isolation in a community sample: its prevalence and correlates with aspects of peoples' lives. Soc Psychiatry Psychiatr Epidemiol 2008 Feb;43(2):140-150. [CrossRef] [Medline]
    70. Wood AM, Linley PA, Maltby J, Kashdan TB, Hurling R. Using personal and psychological strengths leads to increases in well-being over time: a longitudinal study and the development of the strengths use questionnaire. Pers Individ Differ 2011 Jan;50(1):15-19. [CrossRef]
    71. Walach H, Buchheld N, Buttenmüller V, Kleinknecht N, Schmidt S. Measuring mindfulness—the Freiburg mindfulness inventory (FML). Pers Individ Differ 2006 Jun;40(8):1543-1555. [CrossRef]
    72. Green MJ, Myers KR. Graphic medicine: use of comics in medical education and patient care. Br Med J 2010 Mar 3;340:c863. [CrossRef] [Medline]
    73. Williams IC. Graphic medicine: comics as medical narrative. Med Humanit 2012 Jun;38(1):21-27. [CrossRef] [Medline]
    74. McEnery C, Lim MH, Knowles A, Rice S, Gleeson J, Howell S, et al. Development of a moderated online intervention to treat social anxiety in first-episode psychosis. Front Psychiatry 2019;10:581 [FREE Full text] [CrossRef] [Medline]
    75. Rice S, O'Bree B, Wilson M, McEnery C, Lim MH, Hamilton M, et al. Development of a graphic medicine‐enabled social media‐based intervention for youth social anxiety. Clin Psychol 2020 May 5:- (forthcoming). [CrossRef]
    76. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry 2013 Jul;58(7):376-385. [CrossRef] [Medline]
    77. Cuijpers P, Smit F, Bohlmeijer E, Hollon SD, Andersson G. Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. Br J Psychiatry 2010 Mar;196(3):173-178. [CrossRef] [Medline]
    78. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010 Dec;40(12):1943-1957. [CrossRef] [Medline]
    79. Sevilla-Llewellyn-Jones J, Santesteban-Echarri O, Pryor I, McGorry P, Alvarez-Jimenez M. Web-based mindfulness interventions for mental health treatment: systematic review and meta-analysis. JMIR Ment Health 2018 Sep 25;5(3):e10278 [FREE Full text] [CrossRef] [Medline]
    80. Goldberg SB, Tucker RP, Greene PA, Davidson RJ, Wampold BE, Kearney DJ, et al. Mindfulness-based interventions for psychiatric disorders: a systematic review and meta-analysis. Clin Psychol Rev 2018 Feb;59:52-60 [FREE Full text] [CrossRef] [Medline]
    81. Wilson AC, Mackintosh K, Power K, Chan SW. Effectiveness of self-compassion related therapies: a systematic review and meta-analysis. Mindfulness 2018 Oct 24;10(6):979-995. [CrossRef]
    82. Bolier L, Haverman M, Westerhof GJ, Riper H, Smit F, Bohlmeijer E. Positive psychology interventions: a meta-analysis of randomized controlled studies. BMC Public Health 2013 Feb 8;13:119 [FREE Full text] [CrossRef] [Medline]
    83. D'Zurilla TJ, Goldfried MR. Problem solving and behavior modification. J Abnorm Psychol 1971 Aug;78(1):107-126. [CrossRef] [Medline]
    84. Nezu AM. Problem solving and behavior therapy revisited. Behav Ther 2004;35(1):1-33. [CrossRef]
    85. Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol 2001;52:1-26. [CrossRef] [Medline]
    86. Bellamy C, Schmutte T, Davidson L. An update on the growing evidence base for peer support. Ment Health Soc Incl 2017 Jun 12;21(3):161-167. [CrossRef]
    87. Bargas-Avila J, Hornbæk K. Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2011 Presented at: CHI'11; May 7-12, 2011; Vancouver, BC, Canada. [CrossRef]
    88. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods 2002 Mar;7(1):105-125. [CrossRef] [Medline]
    89. Schaub MP, Wenger A, Berg O, Beck T, Stark L, Buehler E, et al. A web-based self-help intervention with and without chat counseling to reduce cannabis use in problematic cannabis users: three-arm randomized controlled trial. J Med Internet Res 2015 Oct 13;17(10):e232 [FREE Full text] [CrossRef] [Medline]
    90. Ali K, Farrer L, Gulliver A, Griffiths KM. Online peer-to-peer support for young people with mental health problems: a systematic review. JMIR Ment Health 2015;2(2):e19 [FREE Full text] [CrossRef] [Medline]
    91. Low KG, Charanasomboon S, Lesser J, Reinhalter K, Martin R, Jones H, et al. Effectiveness of a computer-based interactive eating disorders prevention program at long-term follow-up. Eat Disord 2006;14(1):17-30. [CrossRef] [Medline]
    92. Biagianti B, Quraishi SH, Schlosser DA. Potential benefits of incorporating peer-to-peer interactions into digital interventions for psychotic disorders: a systematic review. Psychiatr Serv 2018 Apr 1;69(4):377-388 [FREE Full text] [CrossRef] [Medline]
    93. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000 Jan;55(1):68-78. [CrossRef] [Medline]
    94. Ryan RM. Facilitating health behavior change and its maintenance: interventions based on self-determination theory. Eur Health Psychol 2008;10(1):2-5 [FREE Full text]
    95. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011 Mar 10;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    96. Bailey E, Rice S, Robinson J, Nedeljkovic M, Alvarez-Jimenez M. Theoretical and empirical foundations of a novel online social networking intervention for youth suicide prevention: a conceptual review. J Affect Disord 2018 Oct 1;238:499-505. [CrossRef] [Medline]
    97. Sündermann O, Onwumere J, Bebbington P, Kuipers E. Social networks and support in early psychosis: potential mechanisms. Epidemiol Psychiatr Sci 2013 Jun;22(2):147-150 [FREE Full text] [CrossRef] [Medline]
    98. Badcock JC, Shah S, Mackinnon A, Stain HJ, Galletly C, Jablensky A, et al. Loneliness in psychotic disorders and its association with cognitive function and symptom profile. Schizophr Res 2015 Dec;169(1-3):268-273. [CrossRef] [Medline]
    99. Lim MH, Gleeson JF, Alvarez-Jimenez M, Penn DL. Loneliness in psychosis: a systematic review. Soc Psychiatry Psychiatr Epidemiol 2018 Mar;53(3):221-238. [CrossRef] [Medline]
    100. Vannucci A, Flannery KM, Ohannessian CM. Social media use and anxiety in emerging adults. J Affect Disord 2017 Jan 1;207:163-166. [CrossRef] [Medline]
    101. Lin L, Sidani JE, Shensa A, Radovic A, Miller E, Colditz JB, et al. Association between social media use and depression among US Young adults. Depress Anxiety 2016 Apr;33(4):323-331 [FREE Full text] [CrossRef] [Medline]
    102. Rideout VJ. The Common Sense Census: Media Use by Tweens and Teens, 2019. Common Sense Media: Age-Based Media Reviews for Families. 2015.   URL: https://www.commonsensemedia.org/research/the-common-sense-census-media-use-by-tweens-and-teens-2019 [accessed 2020-07-21]
    103. Kendal S, Kirk S, Elvey R, Catchpole R, Pryjmachuk S. How a moderated online discussion forum facilitates support for young people with eating disorders. Health Expect 2017 Feb;20(1):98-111 [FREE Full text] [CrossRef] [Medline]
    104. Lederman R, Gleeson J, Wadley G, D’alfonso S, Rice S, Santesteban-Echarri O, et al. Support for carers of young people with mental illness. ACM Trans Comput-Hum Interact 2019 Feb 23;26(1):1-33. [CrossRef]
    105. Donker T, Bennett K, Bennett A, Mackinnon A, van Straten A, Cuijpers P, et al. Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: randomized controlled noninferiority trial. J Med Internet Res 2013 May 13;15(5):e82 [FREE Full text] [CrossRef] [Medline]
    106. Manicavasagar V, Horswood D, Burckhardt R, Lum A, Hadzi-Pavlovic D, Parker G. Feasibility and effectiveness of a web-based positive psychology program for youth mental health: randomized controlled trial. J Med Internet Res 2014 Jun 4;16(6):e140 [FREE Full text] [CrossRef] [Medline]
    107. Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res 2012 Dec 31;15(1):e2 [FREE Full text] [CrossRef] [Medline]
    108. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012 Mar;50(3):217-226 [FREE Full text] [CrossRef] [Medline]
    109. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011 Dec;104(12):510-520 [FREE Full text] [CrossRef] [Medline]
    110. D'Alfonso S, Santesteban-Echarri O, Rice S, Wadley G, Lederman R, Miles C, et al. Artificial intelligence-assisted online social therapy for youth mental health. Front Psychol 2017;8:796 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    AI: artificial intelligence
    CBT: cognitive behavioral therapy
    K10: Kessler 10
    MOST: Moderated Online Social Therapy
    MOST+: Enhanced Moderated Online Social Therapy
    NHMRC: National Health and Medical Research Council
    PHQ-9: Patient Health Questionnaire-9
    SNS: social networking sites


    Edited by G Eysenbach; submitted 21.11.19; peer-reviewed by T Campellone, L Schwab-Reese, J Chu, K Fauvelle; comments to author 29.02.20; revised version received 12.05.20; accepted 13.06.20; published 13.08.20

    ©Mario Alvarez-Jimenez, Simon Rice, Simon D'Alfonso, Steven Leicester, Sarah Bendall, Ingrid Pryor, Penni Russon, Carla McEnery, Olga Santesteban-Echarri, Gustavo Da Costa, Tamsyn Gilbertson, Lee Valentine, Laia Solves, Aswin Ratheesh, Patrick D McGorry, John Gleeson. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.08.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.