This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Mental health disorders are the most prevalent health issues among postsecondary students, yet few solutions to this emerging crisis exist. While mobile health technologies are touted as promising solutions for the unmet mental health needs of these students, the efficacy of these tools remains unclear. In response to these gaps, this study evaluates Thought Spot, a mobile and web app created through participatory design research.
The goal of the research is to examine the impact of Thought Spot on mental health and wellness help-seeking intentions, behaviors, attitudes, self-stigma, and self-efficacy among postsecondary students in Canada.
A 2-armed randomized controlled trial involving students from three postsecondary institutions was conducted. Students were eligible if they were aged 17 to 29 years, enrolled in full-time or part-time studies, functionally competent in English, and had access to a compatible digital device. The usual care group received a mental health services information pamphlet. The intervention group received the Thought Spot app on their digital device. Thought Spot is a standalone app that allows users to add, review, and search crowdsourced information about nearby mental health and wellness services. Users can also track their mood on the app. Outcomes were self-assessed through questionnaires collected at baseline and 3 and 6 months. The primary outcome was change in formal help-seeking intentions from baseline to 6 months, measured by the General Help-Seeking Questionnaire. A mixed-effects model was used to compare the impact of usual care and intervention on the primary outcome (formal help-seeking intentions). Secondary outcomes included changes in informal help-seeking intentions and help-seeking behaviors, help-seeking attitudes, self-stigma, and self-efficacy.
A total of 481 students were randomized into two groups: 240 to usual care, and 241 to the intervention group. There were no significant differences in help-seeking intentions between the usual care and intervention groups over 6 months (
Prompting postsecondary students about mental health and help-seeking appears to increase help-seeking intentions. mHealth interventions may be as effective as information pamphlets in increasing formal help-seeking but may confer a small advantage in driving help-seeking from informal sources. Although there is enthusiasm, developers and health policy experts should exercise caution and thoroughly evaluate these types of digital tools. Future studies should explore the cost-effectiveness of digital interventions and develop strategies for improving their efficacy.
ClinicalTrials.gov NCT03412461; https://clinicaltrials.gov/ct2/show/NCT03412461
RR2-10.2196/resprot.6446
Mental health disorders among postsecondary students are a global public health concern [
The proliferation of mobile devices and their ubiquity in the lives of transition-aged youth has encouraged developers, postsecondary institutions, and health care organizations to focus their efforts on online and mobile health (mHealth) interventions that address the mental health challenges faced by this population [
Applying principles from participatory action and participatory design research [
Features on the Thought Spot mobile app and online platform, designed by students for students to allow users to find and share health, mental health, and wellness resources (spots) using a map-based database of crowdsourced resources and a self-contained search feature.
The team conducted a 2-armed RCT using participants who were students at three postsecondary institutions in the Greater Toronto Area (University of Toronto, Ryerson University, and George Brown College). The study staff and biostatistician were blinded throughout the study. The protocol was approved by the research ethics boards of each participating postsecondary institution and the Centre for Addiction and Mental Health and has been previously published (
Participants were recruited using the most effective methods identified by two focus groups and the Thought Spot youth advisory committee during earlier stages of the project [
Full-time and part-time students aged 17 to 29 years who were enrolled at any of the three postsecondary institutions, who were functionally competent in English, and who had access to a digital device compatible with the intervention were eligible to participate in the study. Active suicidality was the sole exclusion criterion, but no participants met this criterion during the screening process (
Consolidated System of Reporting Trials flow diagram detailing the order of screening, randomization, and follow-up procedures.
Participants were randomly assigned using a 1:1 allocation ratio to either receive access to Thought Spot or to receive a school-specific mental health services information pamphlet. Randomization and allocation was performed using REDCap, a secure, browser-based, electronic data capture system [
The primary outcome was a change in formal help-seeking intentions from baseline to 6 months measured by the General Help-Seeking Questionnaire (GHSQ) [
Statistical analysis was performed by a biostatistician (MS) who was blinded to randomization. To determine the required sample size, power calculations were conducted using the primary outcome, the average formal help-seeking score on the GHSQ [
Univariate analyses were conducted to describe groups at baseline (
Baseline characteristics of participants who underwent randomizationa.
Characteristics | Control (n=240) | Treatment (n=241) | |
|
|||
|
Female | 188 (78.3) | 190 (78.6) |
|
Male | 47 (19.6) | 44 (18.9) |
|
Nonbinary | 5 (2.1) | 7 (2.5) |
Age in years, mean (SD) | 23.2 (3.1) | 22.9 (3.4) | |
|
|||
|
Domestic student | 207 (86.3) | 207 (85.9) |
|
International student | 31 (12.9) | 33 (13.7) |
|
Don’t know | 2 (0.8) | 1 (0.4) |
|
|||
|
High school diploma | 144 (60.0) | 140 (58.1) |
|
College diploma | 23 (9.6) | 9 (3.7) |
|
Bachelor’s degree | 59 (24.6) | 74 (30.7) |
|
Master’s degree | 11 (4.6) | 9 (4.2) |
|
Doctoral degree | 0 | 0 |
|
Other | 3 (1.3) | 9 (3.7) |
|
|||
|
College | 110 (45.8) | 114 (47.3) |
|
University | 130 (54.2) | 125 (51.9) |
|
Did not answer | 0 | 2 (0.8) |
|
|||
|
Yes | 168 (70.0) | 172 (70.8) |
|
No | 55 (23.0) | 63 (26.1) |
|
Don’t know | 16 (6.7) | 6 (4.6) |
|
Did not answer | 1 (0.4) | 0 |
GHSQc formal sources, mean (SD) | 8.6 (3.9) | 8.3 (4.1) | |
GHSQ informal sources, mean (SD) | 36.6 (8.5) | 36.1 (9.4) |
aPercentages may not total 100 due to rounding. There was no significant difference between the trial groups.
bPostsecondary institution was missing for two participants.
cGHSQ: General Help-Seeking Questionnaire.
Similar models were conducted to explore the secondary outcomes of this study except for help-seeking behavior. This outcome was measured using the binary-scaled AHSQ and analyzed using a mixed binomial logistic regression model. The
For
From March 2018 to January 2019, 481 participants were randomized from three Canadian postsecondary institutions into a 6-month trial on a rolling basis. Of these participants, 240 were assigned to the control group and 241 were assigned to the intervention group (
The mixed-effects model found a significant time effect (
An analysis of secondary outcomes found no significant group-by-time interactions for help-seeking intentions from informal sources (GHSQ), help-seeking behavior from formal sources (AHSQ), help-seeking attitudes toward professional help (ATSPPH), self-efficacy (SSOSH) and self-stigma (YES-MH;
Main results from mixed models.
Participant questionnaires | Linear contrasta | Group-by-time interactionb | |||||
|
Estimatea | 95% CI | DFc | ||||
|
|||||||
|
Formal resources | 0.39 | –0.34 to 1.12 | .30 | 0.85 | 2/877 | .43 |
|
Informal resources | 0.36 | –1.21 to 1.92 | .65 | 0.70 | 2/811 | .50 |
GHSQ (per-protocol) – formal resources | 0.18 | –0.62 to 0.98 | .66 | 0.14 | 2/757 | .87 | |
GHSQ (dropout)b – formal resources | 1.35 | –0.51 to 3.21 | .15 | 1.18 | 2/145 | .31 | |
GHSQ (controlled for site)e – formal resources | 0.39 | 0.34 to 1.13 | .29 | 0.85 | 2/877 | .43 | |
ATSPPHf | –0.15 | –0.91 to 0.61 | .70 | 1.39 | 2/850 | .25 | |
SSOSHg | 0.08 | –1.13 to 1.28 | .90 | 0.03 | 2/876 | .97 | |
YES-MHh: self | –0.09 | –0.73 to 0.55 | .79 | 0.12 | 2/801 | .89 | |
YES-MH: service | 0.47 | –0.40 to 1.34 | .29 | 0.60 | 2/823 | .55 | |
YES-MH: system | 0.62 | –0.26 to 1.50 | .17 | 1.31 | 2/818 | .27 | |
YES-MH: total | 0.16 | –1.52 to 1.83 | .85 | 0.08 | 2/738 | .92 |
aThe linear contrast tests the change from baseline to end of the trial across groups (primary outcome); (6 months – baseline) in control minus intervention. Positive values indicate larger increases in the intervention group.
bThe group-by-time interaction tests for any difference in the group trajectories (not primary outcome). Sensitivity analysis controls for all variables at α= .20. These variables are listed in
cDF: degree of freedom.
dGHSQ: General Help-Seeking Questionnaire.
eAn additional sensitivity analysis controlling for site as fixed effects was conducted, but site was not significant and results were similar to those for the main model.
fATSPPH: Attitudes Toward Seeking Professional Psychological Help Scale.
gSSOSH: Self-Stigma of Seeking Help Scale.
hYES-MH: Youth Efficacy/Empowerment Scale–Mental Health.
Main results from binomial models.
Participant questionnaires | Linear contrasta | Group-by-time interactionb | ||||
|
Odds ratio | 95% CI | Wald χ2 | DFc | ||
AHSQd – formal resources | 0.80 | 0.48-1.34 | .39 | 1.3 | 2 | .53 |
AHSQ – informal resources | 0.86 | 0.71-1.02 | .09 | 4.5 | 2 | .11 |
aIn a binomial model, the contrast is the ratio of odds ratios. A ratio lower than 1 indicates a larger increase in the probability of positive answers in the intervention group.
bThe statistic used in binomial models is the chi-square statistic.
cDF: degree of freedom.
dAHSQ: Actual Help-Seeking Questionnaire.
The sex effect was significant for help-seeking behavior related to formal and informal sources, help-seeking attitudes toward professional help, and self-efficacy (
Of the participants in the intervention group, 70.1% (169/241) met the definition of compliance. In comparing these participants to the control group, the per-protocol analysis led to the same conclusion for all primary and secondary outcomes with nonsignificant linear contrast (difference in change 0.18, t711=0.439,
Of the 241 people randomized to the intervention group, 168 visited Thought Spot, resulting in 3696 clicks recorded between March 2018 and June 2019. Overall, users viewed 190 Spots, conducted 293 searches, and created 74 Thoughts. Spots are locations of mental health and wellness resources. Thoughts are users’ personal journal entries on the app. Details of the user data will be published independently of these findings.
In the analysis of our primary outcome, there were no significant differences in the formal help-seeking intentions of postsecondary students between control and intervention groups. However, both groups experienced a similar increase in formal help-seeking intentions during the 6-month study period, as assessed by the GHSQ. These findings suggest that prompting youth about mental health, regardless of the delivery method (eg, information pamphlet or mHealth intervention), may increase help-seeking intentions. Our results are consistent with findings from previous RCTs involving online mental health services, in which the interventions did not lead to significant differences in formal help-seeking compared with an active control group [
Analyses of secondary outcomes revealed no significant group-by-time interactions for help-seeking intentions from informal sources, attitudes toward seeking professional help, self-efficacy, or self-stigma. Although not significant, there is some evidence suggesting a small increase in help-seeking behavior related to informal sources between 3 and 6 months in the intervention group, whereas a small decrease was seen in the control group. This difference contrasts with findings from previous studies of help-seeking interventions, which reported no effect on informal help-seeking [
When looking at the differences between AHSQ scores, the results suggest that females tend to seek help from formal sources more than males. However, when looking at the informal AHSQ scores, males sought more help from informal sources than females. This observation complements the existing literature on gender differences in transition-aged youths’ help-seeking behaviors. Based on a cross-section survey study, Findlay and Sunderland [
The per-protocol analysis indicated that compliant participants increased their help-seeking more than noncompliant participants, which suggests that repeated visits may contribute to changes in help-seeking intentions. While these exploratory findings should be approached with caution, it supports the emerging interest on the impact of repeated app use on study outcomes [
One of the noteworthy strengths of our trial is its standing as one of the few RCTs with large sample sizes that evaluate the effectiveness of mHealth interventions for help-seeking among youth in postsecondary settings [
The study had some limitations. Group assignment could not be blinded for participants. There were also software bugs that led to an inconsistent app environment and usability issues during the trial. These issues could have affected the level of user engagement and compliance with the intervention and ultimately the effectiveness of Thought Spot because some participants had difficulty accessing key functions of the app during certain points of the trial.
There was also a number of participants who did not complete the 3-month survey packages. There were no software bugs or reported issues that could have led to the drop in participation and the reason for this observation is unknown.
Additionally, the effect size for the change in informal help-seeking behavior was small and was noted only in the 3- to 6-month period. Similarly, due to sample size restrictions, we were unable to compare the impact of Thought Spot among international and domestic students. Given the unique barriers associated with attending college in an unfamiliar location [
In summary, there were no significant differences in formal help-seeking intentions between the control and intervention groups. Female participants sought help from formal resources more often than males, whereas males were more likely to seek help from informal sources than females. There was some evidence of a small increase in informal help-seeking behavior between 3 and 6 months in the intervention group. Both groups experienced a similar increase in formal help-seeking intentions over 6 months. These findings suggest a need to further explore the effectiveness of mHealth technologies in supporting the mental health help-seeking needs of transition-aged youth. It is increasingly important as a next step to compare the cost-effectiveness of Thought Spot and information pamphlets for understanding the feasibility and sustainability of mHealth tools compared with existing strategies [
Thought Spot randomized controlled trial protocol and statistical analysis plan.
Automated process for participants self-reporting active suicidality.
Formulas to calculate effect sizes for repeated measures and pretest/posttest research designs.
Group-by-time interaction for help-seeking behavior from informal sources.
Gender effects for help-seeking behaviors, intentions, and attitudes toward professional help and self-efficacy.
Gender effects for help-seeking behaviors, intentions, and attitudes toward professional help and self-efficacy when comparing females and nonbinary to males.
Control variables or interactions that were added to the dropout model aside from the baseline General Help Seeking Questionnaire and sex.
CONSORT-EHEALTH checklist (V 1.6.1).
Actual Help-Seeking Questionnaire
Attitudes Toward Seeking Professional Psychological Help Scale–Short Form
Centre for Addiction and Mental Health
General Help-Seeking Questionnaire
mobile health
odds ratio
randomized controlled trial
Research Electronic Data Capture
Self-Stigma of Seeking Help Scale
Youth Efficacy/Empowerment Scale–Mental Health
This project was supported by eHealth Innovation Partnership Program grant from the Canadian Institute for Health Research (EH1-143558). The Canadian Institute for Health Research was not involved in the study design, data analysis, data interpretation, or writing of the report.
We thank all students who participated in the trial and in our youth engagement activities. We also thank all Centre for Addiction and Mental Health (CAMH) staff, collaborators, youth advisory committee members, and practicum students who were involved and supported the Thought Spot project, including: Daryl Boshart, Almerinda Collela, Paulysha De Gannes, Dr Gunther Eysenbach, Genevieve Ferguson, Olivia Heffernan, Tyson Herzog, Dr Jeffrey Hoch, Dr Huan Jiang, Mandee Kaur, Dr Marcus Law, Julia Roy, Shehab Sennah, Sarah Sharpe, Angie Solomon, Chelsea Stunden, Vivetha Thambinathan, Tim Tripp, Nicole VanHeerwaarden, Julie Zasada, and Hema Zbogar. We thank the CAMH REDCap Operations Committee for their assistance in implementing the e-consent process and survey delivery procedures. QoC Health served as the software developers of the Thought Spot platform.
None of the authors have conflicts of interest to disclose.