This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Mental ill-health is the leading cause of disability worldwide. Moreover, 75% of mental health conditions emerge between the ages of 12 and 25 years. Unfortunately, due to lack of resources and limited engagement with services, a majority of young people affected by mental ill-health do not access evidence-based support. To address this gap, our team has developed a multimodal, scalable digital mental health service (Enhanced Moderated Online Social Therapy [MOST+]) merging real-time, clinician-delivered web chat counseling; interactive user-directed online therapy; expert and peer moderation; and peer-to-peer social networking.
The primary aim of this study is to ascertain the feasibility, acceptability, and safety of MOST+. The secondary aims are to assess pre-post changes in clinical, psychosocial, and well-being outcomes and to explore the correlations between system use, perceived helpfulness, and secondary outcome variables.
Overall, 157 young people seeking help from a national youth e-mental health service were recruited over 5 weeks. MOST+ was active for 9 weeks. All participants had access to interactive online therapy and integrated web chat counseling. Additional access to peer-to-peer social networking was granted to 73 participants (46.5%) for whom it was deemed safe. The intervention was evaluated via an uncontrolled single-group study.
Overall, 93 participants completed the follow-up assessment. Most participants had moderate (52/157, 33%) to severe (96/157, 61%) mental health conditions. All a priori feasibility, acceptability, and safety criteria were met. Participants provided mean scores of ≥3.5 (out of 5) on ease of use (mean 3.7, SD 1.1), relevancy (mean 3.9, SD 1.0), helpfulness (mean 3.5, SD 0.9), and overall experience (mean 3.9, SD 0.8). Moreover, 98% (91/93) of participants reported a positive experience using MOST+, 82% (70/93) reported that using MOST+ helped them feel better, 86% (76/93) felt more socially connected using it, and 92% (86/93) said they would recommend it to others. No serious adverse events or inappropriate use were detected, and 97% (90/93) of participants reported feeling safe. There were statistically significant improvements in 8 of the 11 secondary outcomes assessed: psychological distress (
MOST+ is a feasible, acceptable, and safe online clinical service for young people with mental ill-health. The high level of perceived helpfulness, the significant improvements in secondary outcomes, and the correlations between indicators of system use and secondary outcome variables provide initial support for the therapeutic potential of MOST+. MOST+ is a promising and scalable platform to deliver standalone e-mental health services as well as enhance the growing international network of face-to-face youth mental health services.
Mental ill-health is the number one cause of disability worldwide [
Despite the prevalence and impact of mental illness, between 35% and 57% of people with mental health disorders do not access treatment in high-income countries [
The internet, mobile technologies, and social media have the potential to address the global crisis in the rate at which young people access evidence-based mental health care. Internet-enabled mobile devices are a pervasive element of young people’s lives, with 45% of adolescents being on the web almost constantly [
Currently, there are 4 main types of digital interventions for mental health: self-guided web-based interventions, standalone mental health mobile apps, online peer support groups or interventions, and web-based counseling with registered professionals. Previous trials have shown that the first generation of self-guided web-based interventions, particularly those delivering cognitive behavioral therapy (CBT) and including human support [
The number of mobile apps targeting mental health has grown exponentially over the past few years. According to a 2017 report, almost 500 unique apps were targeting mental health disorders in 2017 [
A growing number of online peer support groups and social networking sites (SNS) exist for people with mental health problems. Overall, the extant evidence suggests that online peer support groups can foster a sense of social connectedness, empowerment, and improved quality of life as well as reduce depression and emotional distress [
The third main type of digital support is web-based counseling (ie, real-time web chat with clinicians). There is initial evidence that web chat is an effective way to deliver mental health support [
Our group has developed a novel and evolving model of web-based behavioral interventions entitled Moderated Online Social Therapy (MOST). The MOST model merges (1) interactive web therapy, (2) peer-to-peer web-based social networking, (3) peer, and (4) clinical moderation. Successive iterations and evolutions of MOST have been successfully adapted for, and trialed with, young people with psychosis [
The overarching aim of this study (trial registration: ACTRN12617000370303) was to determine the feasibility, acceptability, and safety of MOST+ for young people seeking online mental health support. The secondary aims of the project were (1) to assess changes in psychological distress, well-being, depression, stress, social support, loneliness, basic psychological needs (self-competence, relatedness, and autonomy), strengths usage, and mindfulness skills from the point of engagement to post intervention and (2) to explore the associations between system usage, perceived helpfulness, and secondary outcome variables. We hypothesized that MOST+ would be regularly used, favorably received, and safe against a priori established criteria (described in detail in the
The methods of this study have been described in detail elsewhere [
The sample included 157 young people recruited via an opt-in process at the point of entry to eheadspace through a link on the home page. The inclusion criteria were as follows: (1) help-seeking young people with concerns about their own mental health, (2) people aged 16 to 25 years, and (3) people with the ability to provide informed consent and comply with study procedures. The participants who indicated on the web that they understood and consented to the study procedures were recruited into the study.
To ensure the safety of the online social network, some participants were excluded from access to the social networking component of MOST+ (defined as partial access; see under
The mean age of the participants was 19.1 (SD 2.3) years, with 77% female participants. A total of 87% (137/157) of the participants were born in Australia and 11% (17/157) spoke languages other than English. Moreover, 70% (110/157) of the participants were from metropolitan areas, 28% (44/157) from rural areas, and 2% (3/157) from remote areas. In addition, 3% (5/157) of the participants identified themselves as Aboriginals and/or Torres Strait Islanders. Furthermore, 59% (93/157) of the participants had not previously used youth mental health services and 37% (58/157) had never received any mental health support. A total of 57% (89/157) of the participants were engaged in paid work and 77% (121/157) were studying part-time or full-time. The main reasons for seeking help included sadness (38%: 60/157) and anxiety (22%:35/157), followed by feelings of distress (9.6%:15/157). Baseline clinical measures indicated that the majority of participants had mental ill-health. Specifically, the mean baseline Kessler 10 (K10) score was 32.03 (SD 7.72), with 61% (57/93) scoring 30 (indicative of a severe mental health disorder) and 33% (31/93) scoring 25-29 (indicative of a moderate mental health disorder) [
Mean (SD) and within-group effect sizes (Cohen d) for outcome measures (N=93).
Characteristics | Baseline, mean (SD) | Follow-up, mean (SD) | Cohen |
|
K10a | 32.03 (7.680) | 29.43 (8.119) | <.001 | −0.39 (−0.68 to −0.10) |
WEMWSb | 6.58 (2.174) | 7.60 (2.232) | <.001 | 0.51 (0.21 to 0.80) |
PSSc | 10.65 (2.483) | 9.52 (2.940) | <.001 | −0.44 (−0.72 to −0.14) |
PHQ-9d | 15.76 (6.322) | 13.98 (6.514) | .008 | −0.29 (−0.57 to −0.01) |
UCLAe | 9.23 (1.984) | 8.83 (2.224) | .04 | −0.23 (−0.52 to −0.06) |
Competencef | 20.69 (6.449) | 22.27 (6.494) | .005 | 0.30 (0.01 to 0.60) |
Relatednessg | 35.61 (8.900) | 36.85 (7.412) | .08 | 0.17 (−0.12 to 0.46) |
Autonomyh | 25.68 (6.663) | 27.61 (7.148) | .001 | 0.36 (0.07 to 0.65) |
FSi | 10.06 (5.303) | 11.28 (4.935) | .004 | 0.30 (0.01 to 0.59) |
SUSj | 54.20 (16.621) | 56.40 (17.361) | .21 | 0.13 (−0.15 to 0.42) |
FMIk | 28.77 (6.513) | 30.08 (7.184) | .08 | 0.20 (−0.10 to 0.48) |
aK10: Kessler 10.
bWEMWS: 3 items from the Warwick-Edinburgh Mental Well-being Scale.
cPSS: Perceived Stress Scale.
dPHQ-9: Patient Health Questionnaire-9.
eUCLA: UCLA Loneliness Scale (Version 3).
fCompetence: subscale of the Basic Psychological Need Satisfaction Scale.
gRelatedness: subscale of the Basic Psychological Need Satisfaction Scale.
hAutonomy: subscale of the Basic Psychological Need Satisfaction Scale.
iFS: Friendship Scale.
jSUS: Strengths Use Scale.
kFMI: Freiburg Mindfulness Inventory-Short Form.
A large multidisciplinary team of researchers, clinical psychologists, programmers, creative writers, graphic artists, and experts in human-computer interaction worked in collaboration with end users to iteratively develop the MOST+ platform [
MOST+ was conceived as an accessible web-based youth mental health service delivering immediate, short-term, flexible, and evidence-based support to help-seeking young people with mental ill-health. MOST+ was designed to be scalable through the integration of multiple modes of web-based support, thus enabling varying levels of direct support by peer moderators and clinicians.
MOST+ adopted a strengths-based approach [
Psychosocial interventions in MOST+ took the form of brief web-based comics called
Extract of a mindfulness online comic.
Participants with full access were able to communicate with one another and the peer moderators in the
The web chat was fully integrated within MOST+. Young people using the system could request access to a clinician-delivered web chat between 4 PM and 12 AM. This included real-time web counseling focused on reducing immediate distress, supporting positive self-care, and facilitating referral to additional support where appropriate. Following a web chat session and based on the context of the consultation, MOST+ clinicians suggested specific, relevant content from MOST+ (eg, web-based comic,
Participants who consented to the study completed a 15-min web-based survey [
Irrespective of the level of access, all participants were enrolled in the MOST+ intervention for 1 week, with the option to extend their enrollment on a weekly basis over the duration of the intervention period (ie, a minimum of 1 week to a maximum of 9 weeks). The participants were shown a
MOST+ incorporated clinician as well as peer moderation. Clinical moderation primarily focused on ensuring the safety of the social network. Specifically, MOST+ clinicians monitored new contributions to the network for indicators of clinical risk. The social network was moderated by an on-duty MOST+ clinician daily. Safety checks (ie, monitoring any indications of risk on the social network) were undertaken a minimum of 2 times per week day and once daily on weekends and public holidays (see
The
The safety protocol comprised 3 levels of security: (1) system and privacy protection, (2) web safety, and (3) clinical safety. MOST+ had built-in security and data protection to prevent unauthorized access to the platform, which has been described elsewhere [
The MOST+ clinical safety protocol included manual and automated procedures. First, information related to clinical risk (posts or messages) was screened by clinical moderators twice each weekday and daily on weekends and public holidays. Second, MOST+ incorporated an automatic alert system that monitored self-harm–related terms posted on the social feed. Any detected increased risk or inappropriate use activated the safety protocol (
The participants were able to control the extent to which they could be identified by other users within the social network, including whether they used their first name or a nickname and whether their profile picture included a photo. As noted above, following account deactivation, the participants’ accounts and activities were hidden from MOST+. The participants could also choose to
The primary outcome variables were intervention feasibility, acceptability, and safety. All outcomes were assessed at baseline and at follow-up. Baseline assessments were conducted on the web as part of the onboarding process. Follow-up assessment occurred approximately 4 days after the initial account deactivation (ie, 4 days after a participant opted not to renew their account for an additional week). For those participants who maintained active enrollment across the intervention period, follow-up occurred as soon as possible following the conclusion of the pilot. The participants received a short message service notification indicating that their web follow-up survey was due and that they were able to complete survey items either via the web or telephone.
A self-report user feedback questionnaire was developed based on the user experience approach [
Secondary outcome measures included self-report measures of psychological distress, well-being, depression, stress, social support, loneliness, basic psychological needs (self-competence, relatedness, and autonomy), strengths usage, and mindfulness skills (
Acceptability, safety, and perceived helpfulness ratings using Enhanced Moderated Online Social Therapy (N=93).
Questions | Mean (SD) | Median | Values, n (%)a | ||
|
|||||
|
How would you describe your overall experience on MOST+b?c | 3.9 (0.8) | 4 | 91 (98) | |
|
Please rate the helpfulness of using MOST+d | 3.5 (0.9) | 4 | 80 (86) | |
|
Please rate how quickly you were able to find what you needed on MOST+ (ease of use)e | 3.7 (1.1) | 4 | 80 (86) | |
|
Please rate how relevant you found the content on MOST+f | 3.9 (1.0) | 4 | 82 (88) | |
|
|||||
|
Has using MOST+ helped you to better access support from others?g | 3.59 (1.125) | 4 | 79 (85) | |
|
Please rate whether using MOST+ helped you feel betterh | 3.38 (1.03) | 3 | 76 (82) | |
|
Please rate whether using MOST+ helped you feel more socially connectedh | 3.18 (1.15) | 3 | 70 (86) | |
|
Please rate whether you felt safe using MOST+i | 4.43 (0.82) | 5 | 90 (97) |
aNumber of cases responding in the positive range (3 or higher) based on complete responses.
bMOST+: Enhanced Moderated Online Social Therapy.
cItems rated from 1=not at all positive to 5=very positive.
dItems rated from 1=not at all helpful to 5=very helpful.
eItems rated from 1=not at all quickly to 5=very quickly.
fItems rated from 1=not at all relevant to 5=very relevant.
gItems rated from 1=not at all to 5=very much.
hItems rated from 1=not at all safe to 5=very safe.
iItems rated from 1=not at all confidential to 5=very confidential, asked of participants will full access only.
Overview of secondary outcomes and measures used.
Outcomes of interest | Measures | Descriptions |
Psychological distress | K10a | 10-item, widely recommended measure of psychological distress; validated in adolescents [ |
Psychological well-being | WEMWSb | 3 items of the WEMWS are included in the eheadspace Minimum Data Set and assessed in this study: “I’ve been interested in new things,” “I’ve been feeling useful,” and “I’ve been feeling good about myself” [ |
Perceived stress | PSSc | 10-item measure of the degree to which situations in one’s life are appraised as stressful. Widely used, with acceptable psychometric properties [ |
Depression | PHQ-9d | 9-item measure of severity of depression. Validated in psychiatric and primary care populations [ |
Loneliness | UCLAe | 20-item measure assessing how often the respondent feels disconnected from others. Highly acceptable reliability and validity [ |
Basic psychological needs of competence, relatedness, and autonomy | BPNSf | 21-item measure with 3 subscales (competence, autonomy, and relatedness), drawing from self-determination theory [ |
Social support | FSg | 6-item measure of perceived social isolation, with acceptable psychometric properties in the older adult population [ |
Strengths use | SUSh | 14-item measure assessing the extent to which respondents use their strengths, drawing from positive psychology literature [ |
Mindfulness skills | FMIi | 14-item measure of mindfulness. Appropriate for use in contexts where little experience or knowledge of mindfulness can be expected. Acceptable reliability and validity, including in clinical samples [ |
aK10: Kessler 10.
bWEMWS: 3 items from the Warwick-Edinburgh Mental Well-being Scale.
cPSS: Perceived Stress Scale.
dPHQ-9: Patient Health Questionnaire-9.
eUCLA: UCLA Loneliness Scale (Version 3).
fBPNS: Basic Psychological Need Satisfaction Scale.
gFS: Friendship Scale.
hSUS: Strengths Use Scale.
iFMI: Freiburg Mindfulness Inventory-Short Form.
The patterns of intervention use were tracked in real time. Aggregated data from the user feedback questionnaire were compared with the a priori acceptability and safety criteria to determine the success of the pilot. Paired samples
A total of 93 of the 157 participants recruited for the study were contactable and assessed at follow-up. There were no statistically significant differences in any baseline demographic or clinical variables between those who completed the follow-up assessment and those who were lost to follow-up. All a priori indicators of acceptability were met (
A priori set safety criteria were also met. Specifically, no adverse events, inappropriate use, reports by participants, or unlawful entries pertaining to MOST+ were detected during the study. A total of 97% (90/93) of the participants reported feeling safe using MOST+. Moreover, all clinical measures showed a trend toward improved clinical status at follow-up (
Regarding the overall use of MOST+, there were a total of 1058 log-ins during the 9-week study, with 45.2% (71/157) logging in once, 14% (22/157) logging in twice, and 40.8% (64/157) logging in 3 or more times (
Log-ins and individual usage of the main components of Enhanced Moderated Online Social Therapy (N=157) during the pilot study.
Full sample | Characteristics | ||
Site component | Mean (SD) | Range | Percentage, n (%) |
Log-ins | 6.74 (15.21) | 1-103 | 86 (54.8a) |
Posts and comments | 1.14 (4.69) | 0-45 | 21 (14b) |
Steps | 3.73 (9.88) | 0-87 | 78 (49.7c) |
“Do its” | 1.55 (6.53) | 0-74 | 49.9 (31.8d) |
aPercentage of participants with more than 2 log-ins.
bPercentage of participants with more than 1 posts/comments.
cPercentage of participants completing more than 1 step.
dPercentage of participants completing more than 1
Comparison of log-ins and individual use of the main components between participant groups with full access (n=73) and participant group with partial access (n=84).
Variables | Participants with full access (n=73) | Participants with partial access (n=84) | ||||||
|
Mean (SD) | Range | Participants, n (%) | Mean (SD) | Range | Participants, n (%) | ||
Log-ins | 12.34 (18.75) | 1-103 | 58 (79.5a) | 1.87 (2.23) | 1-18 | 30 (33.3a) | −5.10 (155) | <.001 |
Post and comments | 2.46 (6.66) | 0-45 | 22 (30.1b) | N/Ae | N/A | N/A | N/A | N/A |
Steps | 6.52 (13.55) | 0-87 | 49 (67.1c) | 1.30 (3.35) | 1-23 | 29 (34.5c) | −3.41 (155) | .001 |
“Do its” | 2.78 (9.36) | 0-74 | 31 (42.5d) | 0.49 (1.30) | 0-7 | 19 (22.6d) | −2.22 (155) | .03 |
aPercentage of participants with more than 2 log-ins.
bPercentage of participants with more than 1 posts/comments.
cPercentage of participants completing more than 1 step.
dPercentage of participants completing more than 1
eN/A: not applicable.
There were statistically significant improvements between baseline and follow-up assessments, with a small to medium size, in psychological distress (
For those with full access, a secondary analysis revealed that there were significant improvements in psychological distress (
Given the significantly lower system usage and overall retention rate in participants with partial access compared with those with full access, we reported exploratory correlations between system usage, acceptability ratings, and secondary outcome variables for participants who had full access to MOST+. In terms of system usage and acceptability ratings, there were significant correlations between (1) participants reporting that MOST+ helped them feel better and the number of web-based messages between clinicians and young people (Spearman rho, rs=0.53;
To the best of our knowledge, this is the first study to develop and test a multimodal nationwide web-based mental health service for young people experiencing mental ill-health. As such, MOST+ was designed to be an all-in-one digital mental health app merging engaging, evidence-based therapy modules with expert clinician guidance, peer support, social networking, and real-time clinical support. Baseline clinical measures indicated that the majority of participants using MOST+ had moderate (52/157, 33%) to severe mental health conditions (96/157, 61%) and moderate to severe depressive symptoms (130/157, 83%). The results of this study showed that MOST+ was feasible, acceptable, and safe, with all acceptability and safety indicators exceeding the a priori established criteria. The high level of overall satisfaction and perceived helpfulness provided strong support for the relevance of the intervention content and features for help-seeking young people experiencing significant mental ill-health.
Secondary outcome variables showed significant improvements, with small to medium effect sizes, in 8 of the 11 outcomes assessed. These included psychological distress, perceived stress, psychological well-being, depression, loneliness, social support, autonomy, and self-competence. Similarly, the proportions of participants with severe mental health disorders and moderate to severe depression (as indicated by the K-10 and PHQ-9) were significantly lower at follow-up. Although the uncontrolled design of this study did not allow any causal inferences, it was worth noting that there were a number of significant correlations in the expected direction between several indicators of system usage (ie, number of log-ins, number of steps completed, number of
MOST+ was designed as a scalable and efficient online youth mental health service integrating multiple modes of digital therapy, available 24/7, thus catering to individual needs and preferences of young people. The combination of treatment modalities integrated by MOST+ was reported in previous research [
With the purpose of ensuring the safety of a (potentially) population-level social network, participants could be granted either full or partial access (excluded access to web-based social networking) to MOST+. This provided an opportunity to examine the differences in satisfaction levels, perceived safety, usage, and secondary outcomes in relation to the level of access to the system. Interestingly, participants with access to the social network reported significantly higher levels of overall satisfaction, showed higher levels of usage of MOST+, and were more likely to be interviewed at follow-up (60/73, 78%) compared with their counterparts in the partial access group (36/84, 43%). Moreover, although loneliness and autonomy improved significantly in the group with full access, these domains remained unchanged in the partial access group. Taken together, these findings suggested that limiting access to the social network may have thwarted the sense of autonomy and the motivation to use the system and remained in the study in those with partial access. Conversely, having full access to the system and social network may lead to increased autonomy and reduced loneliness, irrespective of whether young people posted or not. It must be noted that there were no significant differences in any baseline variables (including clinical severity and basic psychological needs) between those with partial access and those with full access. These findings were in keeping with the self-determination theory, which posited that environments that addressed the basic psychological needs of autonomy (ie, sense that one’s own behavior is freely chosen and of one’s own volition), relatedness (ie, feelings of safety, belonging, and connectedness in their social interactions), and self-competence enhanced intrinsic motivation [
Web-based social media interventions provide a unique opportunity to address the pervasive rates of social isolation and lack of social support among young people with mental ill-health. For example, young people with psychosis report an average of three lonely days per week [
This study has several limitations. First, the uncontrolled design precluded any causal inferences about the efficacy of MOST+. Second, given that the study was implemented nationally and all assessments were conducted remotely, there was a 40% attrition rate at follow-up, which may have positively biased the results (ie, young people who felt more positively about the intervention may be more likely to be assessed at follow-up). That being said, the reported attrition rate is among the lowest reported by studies evaluating web-based interventions of the equivalent duration via remote assessments (35%-74%) [
The results of this pilot investigation demonstrated that MOST+ is a highly promising and relevant web-based clinical service for young people with clinically significant mental ill-health as it yielded high satisfaction, safety, and perceived helpfulness as well as encouraging improvements in a wide range of clinical and social outcomes. These initial findings provide
The effectiveness and cost-effectiveness of MOST+ will need to be established via controlled evaluations addressing the limitations of this study. For example, MOST+ could be implemented as a national service and evaluated through hybrid trial designs that blend components of clinical effectiveness and implementation research [
The results from this study indicate that MOST+ is a scalable web-based mental health service that enhances the capacity of traditional web counseling services. Future iterations of MOST+ will incorporate artificial intelligence (AI) and machine learning technologies to further enhance the efficiency of the service (eg, via triaging human support as required) as well as the personalization of the intervention [
Finally, in addition to providing mental health support to young people who are not able to access face-to-face care, MOST+ could be integrated with the growing international network of youth mental health services to address wait-list issues, provide continuity of care in between therapy sessions, and offer relapse prevention support after initial treatment response. Meanwhile, MOST+, in its current form, stands to deliver an accessible and scalable web-based mental health service, providing multiple and integrated modalities of web-based support, to cater to the needs of an increasingly growing number of young people with mental ill-health.
Study procedure. MOST+: Enhanced Moderated Online Social Therapy.
Example participant timelines through the Enhanced Moderated Online Social Therapy (MOST+) intervention.
Enhanced Moderated Online Social Therapy (MOST+) intervention safety algorithm.
artificial intelligence
cognitive behavioral therapy
Kessler 10
Moderated Online Social Therapy
Enhanced Moderated Online Social Therapy
National Health and Medical Research Council
Patient Health Questionnaire-9
social networking sites
The authors wish to thank the young people who generously participated in this study. In addition, the authors wish to thank the eheadspace senior management team and clinicians for supporting this study. This work was supported by a grant from the Young and Well Cooperative Research Centre funded by the National Health and Medical Research Council (NHMRC). MA was supported via a Career Development Fellowship (APP1082934) and an Investigator Grant (APP1177235) by NHMRC. The sponsors did not participate in the design or conduct of this study; in the collection, management, analysis, or interpretation of data; in the writing of the manuscript; or in the preparation, review, approval, or decision to submit this manuscript for publication.
None declared.