This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
College students are increasingly reporting common mental health problems, such as depression and anxiety, and they frequently encounter barriers to seeking traditional mental health treatments. Digital mental health interventions, such as those delivered via the Web and apps, offer the potential to improve access to mental health treatment.
This study aimed to review the literature on digital mental health interventions focused on depression, anxiety, and enhancement of psychological well-being among samples of college students to identify the effectiveness, usability, acceptability, uptake, and adoption of such programs.
We conducted a systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (registration number CRD42018092800), and the search strategy was conducted by a medical research librarian in the following databases: MEDLINE (Ovid), EMBASE (Elsevier), PsycINFO (EbscoHost), the Cochrane Library (Wiley), and Web of Science (Thomson Reuters) from the date of inception to April 2019. Data were synthesized using a systematic narrative synthesis framework, and formal quality assessments were conducted to address the risk of bias.
A total of 89 studies met the inclusion criteria. The majority of interventions (71/89, 80%) were delivered via a website, and the most common intervention was internet-based cognitive behavioral therapy (28, 31%). Many programs (33, 37%) featured human support in the form of coaching. The majority of programs were either effective (42, 47%) or partially effective (30, 34%) in producing beneficial changes in the main psychological outcome variables. Approximately half of the studies (45, 51%) did not present any usability or acceptability outcomes, and few studies (4, 4%) examined a broad implementation of digital mental health interventions on college campuses. Quality assessments revealed a moderate-to-severe risk of bias in many of the studies.
Results suggest that digital mental health interventions can be effective for improving depression, anxiety, and psychological well-being among college students, but more rigorous studies are needed to ascertain the effective elements of these interventions. Continued research on improving the user experience of, and thus user engagement with, these programs appears vital for the sustainable implementation of digital mental health interventions on college campuses.
In the last decade, rising rates of college students experiencing symptoms of depression and anxiety have been observed [
Campus counseling centers are well positioned to provide mental health care. However, many counseling centers across the country are underresourced, have difficulty reaching students in need, and operate at full capacity during much of the year [
Digital mental health interventions, such as those delivered via mobile- and Web-based platforms, offer the possibility of treatment to college students with common mental health problems while circumventing many existing barriers to receiving traditional mental health services, including stigma and time [
The evidence base for digital mental health interventions for general adult populations is vast [
However, there have been limitations of these past reviews, as they have focused exclusively on studies that were randomized controlled trials (RCTs). Although a focus on studies with RCT designs allows researchers to evaluate the efficacy and effectiveness of digital mental health interventions, the exclusion of papers reporting on other study designs presents a significant gap in our ability to assess the uptake and adoption of digital mental health interventions for university students (which could be assessed in nonrandomized designs, including single-arm trials in which an intervention is made available to all students on campus). This is particularly important as the full public health potential of these types of interventions is tied not only to clinical efficacy but also to the successful implementation of these programs in real-world settings. Across the board, the implementation and integration of digital health tools into routine care settings has been a challenge. Many have called for testing digital health tools under more pragmatic conditions to maximize the transfer of knowledge from research trials to real-world implementation [
To be included in this review, studies had to (1) examine an intervention that aimed to improve psychological well-being, psychological distress, stress, depressive, and/or anxious symptoms; (2) deliver the intervention via a digital platform (including mobile phone, website, virtual reality systems, and offline computer programs; they could be delivered as an adjunct to face-to-face interventions); (3) include students enrolled in higher education institutions, such as 2-year community colleges, professional schools (eg, medical school and nursing school), 4-year colleges (ie, bachelor’s degree–granting institutions that do not offer graduate degrees), and universities; (4) report outcomes related to psychological well-being, psychological distress, stress, depressive and anxious symptoms, and/or the use and reach of an intervention; and (5) be written in English. In this paper, we use the term
A comprehensive search strategy was developed using keywords and controlled vocabulary to describe university students, depression and anxiety, and digital mental health interventions. The search strategy was adapted to the electronic databases MEDLINE (Ovid), EMBASE (Elsevier), PsycINFO (EBSCOhost), Web of Science (Thomson Reuters), and the Cochrane Library (Wiley). Each database was searched from the date of inception to April 18, 2019. As some relevant journals (ie,
Search results were uploaded into Rayyan, a Web-based software program that allows for reviewers to collaborate during the study selection process [
Two reviewers extracted the data independently from each eligible study using a Web-based extraction form that was piloted and calibrated with all reviewers before formal data extraction. Discrepancies about data extraction were resolved by discussion, and a third reviewer was brought into the discussion if necessary. The data extracted included the study location, study design, type of comparator, type of prevention/treatment, type of technology, name of technology/program, type of program, primary intervention target(s), presence of support, student population, setting, sample size, length of intervention, usability and acceptability outcomes, uptake and adoption outcomes, psychological outcomes, and type of analyses performed (ie, completer or intent to treat).
This review examined the effectiveness, usability, acceptability, uptake, and adoption of digital mental health interventions for treating depression and anxiety and for enhancing psychological well-being among college students. The effectiveness outcomes included measures of depressive symptomatology (eg, Beck Depression Inventory-II [
For the purpose of this review, usability was defined as the quality of a user’s experience when interacting with a program. Usability is an umbrella term that includes the ease of learning a program, the efficiency of use, the memorability of it, and the subjective satisfaction with a program. The usability outcomes include standard usability measures (eg, the System Usability Scale [
For the purpose of this review, the terms
As this review included both randomized trials and nonrandomized trials, the risk of bias was assessed using 2 separate tools: the Cochrane Collaboration’s tool for assessing risk of bias in randomized trials [
A systematic narrative framework was used to synthesize the data [
A total of 6428 article titles and abstracts were reviewed. Then, 187 full-text articles were reviewed for inclusion, with 89 studies included in the review for data extraction. See
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.
Of the 89 studies included in this review, 72 employed randomized study designs [
We examined if each study appeared to be designed specifically for students (eg, the purpose of the study was focused on college student mental health) or if college students appeared to be used as a convenience sample. As seen in
The majority of these studies (n=46 [
A total of 71 studies utilized a Web-based technology ([
The majority of interventions studied offered some level of support or guidance to users—many of these interventions featured coaching from a human (n=33 [
As seen in
Of the 42 studies deemed effective in producing beneficial changes in the main psychological outcome variables, 36 of those studies utilized a control condition; although the quality of control conditions varied broadly and ranged from treatment as usual or waitlists to other digital programs or face-to-face treatment [
Approximately half of the studies included in this review (n=45 [
As seen in
The vast majority of studies did not specify the size of the pool of potential participants from which the study participants were drawn (n=81 [
Relatively few studies examined the implementation of a digital mental health intervention on a college campus and reported on the implementation outcomes [
As seen in
As seen in
Of the 16 nonrandomized studies screened for risk of bias, the majority of studies (n=10 [
Risk of bias for randomized studies.
Authors and year of publication | Dmna #1b | Dmn #2c | Dmn #3d | Dmn #4e | Dmn #5f | Overall risk |
Alvarez et al, 2008 [ |
LRg | LR | LR | LR | LR | LR |
Arpin-Cribbie et al, 2012 [ |
LR | LR | LR | SCh | LR | SC |
Asbury et al, 2018 [ |
SC | SC | LR | LR | LR | SC |
Auyeung & Mo, 2018 [ |
LR | LR | LR | LR | LR | LR |
Bedford et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Booker & Dunsmore, 2017 [ |
LR | SC | SC | LR | LR | SC |
Braithwaite & Fincham, 2009 [ |
LR | LR | LR | LR | LR | LR |
Braithwaite & Fincham, 2007 [ |
SC | LR | SC | LR | LR | SC |
Buglione et al, 1990 [ |
SC | SC | SC | LR | LR | HRi |
Chiauzzi et al, 2008 [ |
LR | LR | LR | SC | LR | SC |
Cohen et al, 1999 [ |
LR | LR | LR | LR | LR | LR |
Cukrowicz & Joiner, 2007 [ |
SC | LR | LR | LR | LR | SC |
Day et al, 2013 [ |
LR | LR | HR | SC | LR | HR |
Ellis et al, 2011 [ |
LR | LR | LR | SC | LR | LR |
Eustis et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Fernandez et al, 1986 [ |
LR | LR | LR | LR | SC | SC |
Fitzpatrick et al, 2017 [ |
LR | LR | LR | SC | LR | SC |
Flett et al, 2019 [ |
LR | LR | LR | LR | LR | LR |
Frazier et al, 2015 [ |
LR | LR | LR | SC | LR | SC |
Freeman et al, 2008 [ |
SC | LR | HR | HR | LR | HR |
Frith & Loprinzi, 2017 [ |
LR | LR | LR | LR | LR | LR |
Fulmer et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Geisner et al, 2015 [ |
LR | LR | LR | SC | LR | LR |
Gibbel, 2010 [ |
LR | SC | LR | LR | LR | SC |
Grassi et al, 2011 [ |
LR | LR | SC | SC | LR | SC |
Greer, 2015 [ |
LR | LR | LR | LR | LR | LR |
Guille et al, 2015 [ |
LR | LR | LR | LR | LR | LR |
Hall et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Harrer et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Harris et al, 2002 [ |
HR | LR | HR | SC | LR | HR |
Hintz et al, 2015 [ |
LR | LR | LR | SC | LR | SC |
Hoppitt et al, 2014 [ |
LR | LR | LR | LR | LR | LR |
Howell et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Kenardy et al, 2003 [ |
LR | LR | LR | SC | LR | SC |
King et al, 2015 [ |
LR | LR | LR | SC | LR | SC |
Koydemir & Sun-Selisik, 2016 [ |
SC | SC | HR | SC | LR | HR |
Kvillemo et al, 2016 [ |
LR | LR | LR | LR | LR | LR |
Lee & Jung, 2018 [ |
LR | LR | LR | LR | LR | LR |
Levin et al, 2014 [ |
LR | LR | LR | SC | HR | HR |
Levin et al, 2016 [ |
LR | LR | LR | LR | LR | LR |
Levin et al, 2017 [ |
LR | LR | LR | SC | LR | SC |
Lintvedt et al, 2013 [ |
SC | LR | LR | SC | LR | SC |
Mailey et al, 2010 [ |
LR | LR | LR | SC | LR | SC |
Mak et al, 2015 [ |
LR | LR | LR | SC | LR | SC |
Mak et al, 2017 [ |
LR | LR | LR | LR | SC | SC |
McCall et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Melnyk et al, 2015 [ |
SC | LR | LR | SC | SC | HR |
Mogoaşe, 2013 [ |
LR | LR | LR | LR | LR | LR |
Morris et al, 2016 [ |
LR | LR | LR | SC | LR | SC |
Mullin et al, 2015 [ |
SC | LR | LR | SC | LR | SC |
Musiat et al, 2014 [ |
LR | LR | LR | LR | LR | LR |
Nguyen-Feng et al, 2015 [ |
LR | LR | LR | SC | LR | SC |
Nguyen-Feng et al, 2016 [ |
LR | LR | LR | LR | SC | SC |
Nguyen-Feng et al, 2017 [ |
LR | LR | LR | LR | LR | LR |
Nordmo et al, 2015 [ |
SC | LR | LR | LR | LR | SC |
Orbach et al, 2007 [ |
LR | LR | SC | LR | SC | SC |
Radhu et al, 2012 [ |
LR | LR | LR | SC | LR | SC |
Rasanen et al, 2016 [ |
LR | LR | LR | SC | LR | SC |
Richards & Timulak, 2013 [ |
LR | SC | SC | LR | HR | HR |
Richards et al, 2013 [ |
LR | LR | LR | LR | LR | LR |
Richards et al, 2016 [ |
LR | LR | LR | SC | LR | SC |
Rose et al, 2013 [ |
SC | LR | LR | LR | LR | SC |
Sagon et al, 2018 [ |
SC | LR | LR | LR | LR | SC |
Saleh et al, 2018 [ |
SC | LR | LR | LR | LR | SC |
Santucci et al, 2014 [ |
LR | LR | LR | LR | LR | LR |
Sarniak, 2009 [ |
SC | SC | LR | LR | LR | SC |
Seligman et al, 2007 [ |
SC | LR | LR | SC | LR | SC |
Stallman et al, 2018 [ |
LR | LR | LR | LR | LR | LR |
Taitz, 2011 [ |
SC | SC | SC | LR | LR | HR |
Tillfors et al, 2008 [ |
LR | LR | LR | LR | LR | LR |
Villani & Riva, 2008 [ |
LR | LR | LR | LR | LR | LR |
Yang et al, 2015 [ |
LR | LR | LR | SC | LR | SC |
aDmn: domain.
bBias arising from the randomization process.
cBias due to deviations from intended interventions.
dBias due to missing outcome data.
eBias in measurement of the outcome.
fBias in selection of the reported result.
gLR: low risk.
hSC: some concerns.
iHR: high risk.
Risk of bias for nonrandomized studies.
Authors and year of publication | Dmna #1b | Dmn #2c | Dmn #3d | Dmn #4e | Dmn #5f | Dmn #6g | Dmn #7h | Overall risk |
Benton et al, 2016 [ |
MRi | LRj | SRk | LR | SR | SR | LR | SR |
Carey et al, 2016 [ |
MR | LR | LR | LR | LR | SR | LR | SR |
Finlay-Jones et al, 2016 [ |
MR | LR | LR | LR | MR | SR | LR | SR |
Haas et al, 2008 [ |
LR | N/Al | N/A | N/A | N/A | N/A | N/A | LR |
Horgan et al, 2013 [ |
MR | LR | LR | LR | SR | SR | LR | SR |
Kaczmarek et al, 2013 [ |
LR | N/A | N/A | N/A | N/A | N/A | N/A | LR |
Kim et al, 2011 [ |
LR | N/A | N/A | N/A | N/A | N/A | N/A | LR |
Levin et al, 2015 [ |
MR | LR | LR | LR | MR | SR | LR | SR |
Moir et al, 2015 [ |
MR | LR | SR | LR | NIm | SR | LR | NI |
North et al, 2002 [ |
MR | LR | LR | LR | LR | SR | LR | SR |
Palacios et al, 2018 [ |
MR | LR | LR | LR | MR | SR | LR | SR |
Sharry et al, 2013 [ |
MR | LR | LR | LR | MR | SR | LR | SR |
Spadaro & Hunker, 2016 [ |
MR | LR | LR | LR | LR | SR | LR | SR |
Trockel et al, 2011 [ |
CRn | LR | LR | LR | SR | SR | LR | SR |
Williams et al, 2014 [ |
MR | LR | LR | LR | LR | LR | LR | MR |
Wilson et al, 1991 [ |
MR | CR | LR | LR | LR | SR | CR | CR |
aDmn: domain.
bBias due to confounding.
cBias in selection of participants into the study.
dBias in classification of interventions.
eBias due to deviations from intended interventions.
fBias due to missing data.
gBias in measurement of outcomes.
hBias in selection of the reported result.
iMR: moderate risk.
jLR: low risk.
kSR: serious risk.
lN/A: not applicable. Any study deemed low risk in Domain #1 is considered low risk as a whole; thus, other domains are N/A.
mNI: no information.
nCR: critical risk.
This study aimed to synthesize the literature on the effectiveness, usability, acceptability, uptake, and adoption of digital mental health interventions for (1) treating depression and anxiety and (2) enhancing psychological well-being among college students. In doing so, the types of interventions that have been developed and tested were characterized. The vast majority of included studies reported that the digital mental health interventions of interest were either effective, or partially effective, in producing beneficial changes in the main psychological outcome variables. This is consistent with past meta-analyses on digital mental health programs for college students [
The majority of programs were studied on university campuses and enrolled broad samples of undergraduate and graduate students. The focus on universities was not surprising, as many studies were conducted at the university with which the researchers were affiliated, likely because of a combination of ease and investment in one’s own community. Fewer studies took place within health professional (eg, medical school and nursing school) programs. It was notable that only 1 study comprised a community college sample, as it is widely recognized that community college students have higher rates of unmet mental health needs compared with students in traditional 4-year colleges and universities [
College students are often used as a convenience sample for psychological research [
Similar to what has been observed in digital mental health intervention programs for general adult populations [
Although user-centered design can produce programs that are more engaging and enjoyable for users, design principles alone are unlikely to produce interventions that are sustainably used on college campuses. The research-to-practice gap for digital mental health interventions is increasingly being recognized, and leaders in the field have proposed strategies to routinely incorporate implementation science methods into the study of digital mental health interventions [
This study should be interpreted in light of its strengths and limitations. Consistent with best practices, the articles were reviewed by 2 independent reviewers and risk of bias was assessed. The moderate-to-severe risk of bias found in many of the included randomized and nonrandomized trials indicates that the results reported may be biased in favor of the digital mental health tools and should be evaluated in that context. Bias primarily emerged because the outcomes were self-reported in nature and the participants were aware of the intervention they received—2 issues that are exceedingly common in digital health research. Although the search strategy was developed with an experienced research librarian and an additional handsearch was used, it is possible that some relevant publications were missed in the search. Several reviewed studies used active controls or comparison interventions that produced similar effects to the intervention of interest, so we were unable to evaluate the effectiveness of intervention ingredients to inform what components (eg, features or techniques) are relevant for achieving behavior change. Without the gold standard interventions in digital health for college students that could serve as comparisons with newly developed interventions, several studies that were reviewed used active controls or comparison interventions that produced similar effects to the intervention of interest. In addition, none of the included studies utilized noninferiority analyses. Therefore, the true efficacy of most of the interventions was unclear.
Another strength is that we did not limit this review to RCTs of computer- and Web-based programs. As such, this study expands on past work by offering a much broader look at the types of digital mental health programs that have been available for students and a look at the uptake and adoption of such interventions. Uptake and adoption could not have been meaningfully examined if this review was limited to RCTs. However, the consequence of including multiple trial designs precluded us conducting a meta-analysis because of the heterogeneity of the data included.
Digital mental health interventions for depression, anxiety, and the enhancement of psychological well-being have the potential to improve the mental health of college students around the world. The majority of interventions have focused on Web-based technologies, and there remains a need for further research on interventions delivered via mobile phones. To date, published studies on digital mental health programs have primarily been focused on establishing efficacy and/or effectiveness rather than on supporting program uptake and adoption across campus communities. For these programs to realize their potential, they need to be successfully and sustainably implemented on college campuses as part of the array of available mental health services. Further research on digital mental health interventions for college students should focus on designing and testing programs that are viewed as usable and acceptable to students and on methods of implementing such programs on college campuses.
Search strategy overview.
Study details.
Study results.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
randomized controlled trial
EGL is supported by a research grant K08 MH112878 from the National Institute of Mental Health. AKG is supported by a research grant K01 DK116925 from the National Institute of Diabetes and Digestive and Kidney Diseases.
None declared.