Published on in Vol 24 , No 12 (2022) :December

Preprints (earlier versions) of this paper are available at, first published .
Challenges in Recruiting University Students for Web-Based Indicated Prevention of Depression and Anxiety: Results From a Randomized Controlled Trial (ICare Prevent)

Challenges in Recruiting University Students for Web-Based Indicated Prevention of Depression and Anxiety: Results From a Randomized Controlled Trial (ICare Prevent)

Challenges in Recruiting University Students for Web-Based Indicated Prevention of Depression and Anxiety: Results From a Randomized Controlled Trial (ICare Prevent)

Original Paper

1Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

2Department of Mental Health and Prevention, Trimbos Institute, Netherlands Institute for Mental Health and Addiction, Utrecht, Netherlands

3Department of Research and Business Development, HumanTotalCare, Utrecht, Netherlands

4Department of Psychology and Centre for Urban Mental Health, University of Amsterdam, Amsterdam, Netherlands

5Department of Clinical Psychology and Psychotherapy, Technical University Dresden, Dresden, Germany

6Department of Sport and Health Science, Technical University Munich, Munich, Germany

7Department of Clinical Psychology and Psychotherapy, Friedrich Alexander University, Erlangen-Nuremberg, Germany

8International Institute for Psychotherapy, Babeș-Bolyai University, Cluj-Napoca, Romania

9Department of Psychiatry, Amsterdam UMC, location VUmc, Amsterdam, Netherlands

Corresponding Author:

Felix Bolinski, MSc

Department of Clinical, Neuro, and Developmental Psychology

Amsterdam Public Health Research Institute

Vrije Universiteit Amsterdam

De Boelelaan 1105

Amsterdam, 1081HV


Phone: 31 0205985204


Background: Depression and anxiety are common mental health conditions in college and university student populations. Offering transdiagnostic, web-based prevention programs such as ICare Prevent to those with subclinical complaints has the potential to reduce some barriers to receiving help (eg, availability of services, privacy considerations, and students’ desire for autonomy). However, uptake of these interventions is often low, and accounts of recruitment challenges are needed to complement available effectiveness research in student populations.

Objective: The aims of this study were to describe recruitment challenges together with effective recruitment strategies for ICare Prevent and provide basic information on the intervention’s effectiveness.

Methods: A 3-arm randomized controlled trial was conducted in a student sample with subclinical symptoms of depression and anxiety on the effectiveness of an individually guided (human support and feedback on exercises provided after each session, tailored to each participant) and automatically guided (computer-generated messages provided after each session, geared toward motivation) version of ICare Prevent, a web-based intervention with transdiagnostic components for the indicated prevention of depression and anxiety. The intervention was compared with care as usual. Descriptive statistics were used to outline recruitment challenges and effective web-based and offline strategies as well as students’ use of the intervention. A basic analysis of intervention effects was conducted using a Bayesian linear mixed model, with Bayes factors reported as the effect size.

Results: Direct recruitment through students’ email addresses via the central student administration was the most effective strategy. Data from 35 participants were analyzed (individually guided: n=14, 40%; automatically guided: n=8, 23%; care as usual: n=13, 37%). Use of the intervention was low, with an average of 3 out of 7 sessions (SD 2.9) completed. The analyses did not suggest any intervention effects other than anecdotal evidence (all Bayes factors10≤2.7).

Conclusions: This report adds to the existing literature on recruitment challenges specific to the student population. Testing the feasibility of recruitment measures and the greater involvement of the target population in their design, as well as shifting from direct to indirect prevention, can potentially help future studies in the field. In addition, this report demonstrates an alternative basic analytical strategy for underpowered randomized controlled trials.

Trial Registration: International Clinical Trials Registry Platform NTR6562;

International Registered Report Identifier (IRRID): RR2-10.1186/s13063-018-2477-y

J Med Internet Res 2022;24(12):e40892




Offering transdiagnostic prevention programs for depression and anxiety to college and university students (henceforth denoted as students) is relevant for a number of reasons: these conditions are highly comorbid [1,2] and often emerge during the time individuals enter tertiary education [3-6]. Moreover, symptoms of depression and anxiety are highly correlated with various stressors specifically affecting students, such as considerable academic demands [7,8]; balancing life, studies, and student jobs [9,10]; and, more recently, the global COVID-19 pandemic and the measures implemented to reduce its spread [11,12]. In addition, some studies have reported that financial concerns, in particular student debt, might negatively affect students’ mental health [13,14].

Results from the World Health Organization World Mental Health International College Student initiative [15] suggest that approximately one-third of the almost 14,000 surveyed students had had a mental health disorder in the previous year, with major depressive disorder (MDD; 18.5%) and generalized anxiety disorder (GAD; 16.7%) being the most prevalent conditions [5]. Both disorders cause adverse effects on quality of life [16] and can lead to severe interpersonal impairment. For example, research indicates that more than half of the first-year students who presented with depression or anxiety reported severe disturbances in their social lives and close relationships [17]. In addition, mental health complaints negatively affect students’ academic performance and increase dropout [18,19]. A recent study among Dutch adolescents also highlighted the economic costs of subclinical depression, including health care and societal costs related to school absenteeism [20]. Considering the importance of higher education for economic growth [21], preventing the onset of mental health conditions in students matters from an individual, societal, and economic perspective.

In particular, indicated prevention approaches [22] that focus on individuals with subclinical complaints could be beneficial. These have the potential to prevent the onset of, for example, MDD [23] and—in contrast to universal approaches—reduce expenditure on costly treatment by allocating scarce health care resources to those students who are in immediate need [24]. Meta-analyses of community samples suggest that preventive interventions can reduce the incidence rate of depression by approximately 19% [25] and of anxiety by approximately 43% [26].

Despite the need for and availability of such interventions [27,28], uptake of preventive programs is particularly low compared with treatment for psychological disorders [29,30], likely as perceived symptoms are not severe enough yet to motivate help seeking. Students in particular often do not seek professional help—merely approximately a quarter of World Health Organization International College Student Initiative respondents indicated that they would definitely seek treatment for mental health complaints; of those who would not seek help, approximately half indicated that they would rather deal with the problem on their own or preferred to consult with friends or relatives [31]. Fear of stigmatization is another reason for low help-seeking behavior [32,33]. In addition, on-campus mental health services often lack the capacity to meet the needs of those students who seek help [34]. Moreover, often these services focus exclusively on study-related issues (eg, test anxiety and procrastination). As a result, studies suggest that only approximately one-third to one-sixth of students receive adequate help for their complaints [35,36].

Digital interventions provided via the internet on computers or via smartphone apps have been proposed as a way to overcome barriers to the availability of counseling services and help seeking by offering the privacy and autonomy desired by students [37]. A recent meta-analysis on the effectiveness of such interventions for mental health complaints included 48 randomized controlled trials (RCTs) on individually guided (ie, tailored human support) and unguided or automatically guided (ie, computer-generated standardized feedback) digital interventions, most of them based on cognitive behavioral therapy [28]. Small but significant differences favoring the interventions were found only when these were compared with passive controls (eg, depression: g=0.18; anxiety: g=0.27), with individual guidance having no significant effect on these results. However, slightly larger effects on depression outcomes (g=0.29) were found in studies that targeted subclinical complaints. Importantly, the prediction intervals in this meta-analysis suggest that future trials will likely include nil effects [28]. In addition, transdiagnostic components that target both conditions may be beneficial, particularly for depression outcomes (g=0.22) [38]. Although this indicates some potential for indicated transdiagnostic prevention efforts, the general focus on reduction of symptoms rather than prevention of (future) mental health conditions implies a need for further research [39]. However, an RCT has shown that a digital intervention can also prevent the onset of MDD in the general population [23].


On the basis of these considerations, ICare Prevent—a transdiagnostic individually tailored digital intervention for the indicated prevention of depression and anxiety—was developed. We planned to conduct an RCT on the effectiveness of the intervention among students in the Netherlands. However, despite 2 years of extensive countrywide recruitment efforts, this trial was concluded without reaching the targeted sample size (N=252). Therefore, the aims of this study are 3-fold, namely to (1) describe the recruitment process for the RCT, (2) describe participants’ use of the intervention, and (3) conduct a basic analysis of intervention effects on depressive and anxiety symptoms.

General Study Design and Inclusion and Exclusion Criteria

This trial was registered in the International Clinical Trials Registry Platform (NTR6562), and a detailed protocol for the planned trial has been published [40]. In summary, the design entailed a 3-arm parallel superiority trial comparing an individually guided and an automatically guided version of the intervention with care as usual (CAU). The aim was to include 252 students (84 per condition; expected effect size: d=0.35; for the power calculation, see the protocol by Bolinski et al [40]). Recruitment started in June 2017 and was concluded in July 2019.

Dutch- and English-speaking students aged ≥16 years with subclinical symptoms of depression or anxiety, defined as a score of ≥16 on the Center for Epidemiological Studies Depression Scale [41] or ≥5 on the 7-item Generalized Anxiety Disorder Scale (GAD-7) [42], respectively, could participate if they provided written informed consent. They were excluded if they were in remission of a major depressive episode; currently meeting diagnostic criteria for a mood or anxiety, lifetime bipolar, or psychotic disorder; or presenting moderate to high suicide risk (all assessed by phone using the Mini International Neuropsychiatric Interview [MINI]) [43]. In addition, they were excluded if they reported currently being on a waitlist for or having received psychotherapy in the previous half year as well as participating in similar intervention studies at the time of the screening assessment.

Randomization was performed at the individual level at a 1:1:1 allocation ratio following baseline assessment. It was stratified by the type of subclinical symptoms (ie, depression, anxiety, or both according to the MINI). Subsequent assessment points were the midway point (5 weeks after randomization), posttest assessment (8 weeks after randomization), and the 6- and 12-month follow-ups.


Participants first received information on the study. Following registration and provision of written informed consent, self-reported inclusion and exclusion criteria were assessed through web-based questionnaires, followed by the assessment of diagnostic exclusion criteria by phone. Participants were then randomized according to incoming informed consent forms by an independent researcher using a dedicated randomization platform. Owing to the nature of the study, blinding of participants and of the coaches who provided guidance in the individually guided intervention arm was not possible. However, assessors performing the MINI interviews at the posttest and follow-up assessments were blind to the participants’ group allocation (see the flowchart in Figure 1).

Figure 1. Study flowchart (*=1 participant excluded because of dropout after randomization).
View this figure

Recruitment Methods

A broad array of recruitment measures was used, such as social media campaigns, printed advertisements, a paid participant platform, and other project collaborations. Examples of some of the recruitment materials are provided in Multimedia Appendices 1 and 2.

Social Media

Paid social media campaigns on Facebook and Instagram were targeted to higher-education students in the Netherlands. These advertisements contained images that reflected the study aim, a brief outline of the web-based intervention, and a call to visit the study website and register for participation. In addition, information on the study and the call to participate were posted on Facebook groups relevant to students.

Print and News Media

Flyers, posters, and stickers were distributed at universities countrywide. Similar to social media posts, they contained brief information on the study and the intervention as well as contact details (study website and email address). This material was also published in 2 of the largest student newspapers.

Targeted Email Distribution Through Project Collaboration (UvAcare)

In March 2019, recruitment was extended to a student mental health project conducted at another university (UvAcare) [44]. Therein, all students were screened for mental health complaints, including those with more severe presentations of such problems. Screening generally took place at the beginning of a semester. A data-sharing agreement was set up, and data on demographic variables and primary mental health outcomes of a subsample of participating (PhD) students fulfilling the aforementioned inclusion criteria were provided. This collaboration offered the possibility of using students’ email addresses—sent through the central administration—for providing information on the study and the intervention and a digital rather than written informed consent form. In this project collaboration, the use of email addresses remained the primary recruitment channel. A full description of this project is provided in the associated study protocol [44].

Participant Platform

A paid platform for participation in clinical trials was used [45]. The platform maintains a directory of users who are interested in participating in clinical trials. Information on the study was uploaded to the platform, and interested users could sign up to receive the information letter and informed consent form. A brief questionnaire on whether a user was currently registered at a Dutch higher education institution and not currently receiving psychological treatment was implemented as an initial screener.


Key individuals at the university (eg, student bodies, teachers, and counselors) were approached to create support for the study. It was then pitched in classrooms and web-based education portals. Awareness of the project was also created by participating in the largest running event in the Netherlands and wearing a shirt printed with the study website [46]. Local and international conferences were used to generate a network of contacts that could help in the dissemination of project information and, thus, recruitment.


ICare Prevent is a web-based and mobile-supported intervention for the indicated prevention of depression and anxiety. It uses both transdiagnostic and individually tailored elements and was originally developed for the German-speaking general population following existing evidence- and cognitive behavioral therapy–based web-based modules from different digital interventions [23,47]. The intervention comprises a sequential (ie, a session has to be completed to unlock the next) 7-session web-based program with 1 booster session. Participants were advised to complete between 1 and 2 sessions per week, with each requiring approximately 45 to 60 minutes to be completed. In addition, 8 elective modules (on sleep, perfectionism, gratitude, self-esteem, alcohol use, relaxation, acceptance, and rumination) and 5 diaries (positive activities, negative thoughts, sleep, challenging situations, and alcohol use) targeted factors common to both mood and anxiety problems. All the sessions and elective modules included individual exercises that needed to be completed. In line with its transdiagnostic, individually tailored approach, users received information targeting both conditions (sessions 1-4) before they could prioritize the techniques that focused on their most prominent complaints (sessions 5 and 6).

In the individually guided arm, clinical psychology students motivated participants to continue with the sessions and provided structured and personalized feedback on the homework exercises. Those allocated to the automatically guided intervention received standard and computerized feedback after completing each session, which was geared toward motivating them to continue to the next session. Technical and usability questions could be asked by participants in either group.

As part of the study, the intervention was translated and adapted to the student context in the Netherlands. To do this, the first author worked together with a graphic designer and text editor experienced in web editing. Finally, the intervention was tested on spelling and functionality by a number of students who were employed as research assistants. Adaptations included changing the testimonials to be more representative of a diverse student population, changing the focus of exercises to problems more applicable to students, and considerably shortening the amount of text in the modules while keeping the structure and main content of the original intervention intact (see Multimedia Appendix 3 and the protocol by Bolinski et al [40] for a description of the intervention content per session and its transdiagnostic components). Students in the CAU condition were informed that they could seek help from their general practitioner (GP) or student psychologist for any mental health complaints. However, students in all trial arms were free to access CAU during the study.

Primary Outcome Variables

In total, 2 primary outcome variables were assessed by interviewers via telephone to measure changes in disorder-specific symptoms from baseline to posttest and follow-up assessments [40]. For depression, the clinician-rated version of the Quick Inventory of Depressive Symptomatology (QIDS-CR) [48] was used. This 16-item instrument assesses 9 symptom criteria for depression (sad mood, concentration difficulties, self-criticism, suicidal thoughts, general interest, fatigue, sleep, appetite, and psychomotor retardation) based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition [49], with total scores ranging from 0 to 27. Higher scores indicate greater psychopathology. The QIDS-CR has well-established psychometric properties and is sensitive to the effects of treatment [50,51]. It has been shown to generate reliable scores when administered over the phone compared with in-person or paper-and-pencil versions [52]. Anxiety was assessed using the Structured Interview Guide for the Hamilton Anxiety Rating Scale (SIGH-A) [53,54]. A total of 14 items cover anxiety symptoms, resulting in a total score ranging from 0 to 56, with higher scores again indicating greater psychopathology. The SIGH-A has well-established psychometric properties and has provided sensitive diagnoses compared with, for example, in-person ratings [53,55,56]. More information on the instruments has been provided in the study protocol [40].

Other Variables


The original study protocol [40] lists a number of secondary variables. In this report, only data collected through the project collaboration (UvAcare) could be analyzed. Therefore, only a subset of secondary variables that had also been collected in the UvAcare project is reported here.


In addition to during screening, the MINI was also completed over the phone during follow-up assessments. The interview has reasonably good psychometric properties [43,57,58]. In addition, 2 self-report questionnaires were administered at all assessment points after screening. The 9-item Patient Health Questionnaire (PHQ-9) [59] measures the presence of depressive symptoms during the past 2 weeks on a 4-point Likert scale (0=not at all to 3=nearly every day). The PHQ-9 is a reliable and valid screening instrument with established cutoff scores of 5, 10, and 20 referring to mild, moderate, and severe depression, respectively [59]. The GAD-7 is scored identically and measures anxiety symptoms during the past 2 weeks, with total values of 5, 10, and 15 indicating mild, moderate, and severe manifestations of generalized anxiety, respectively [42]. The questionnaire has good psychometric properties [60].

Health Care Use

At posttest assessment and during follow-up assessments, CAU use was monitored by asking participants if they had been in contact with any health care professionals (eg, GP, student counseling services, or psychologist) in the previous 2 months. If so, they were asked to indicate how often they had had contact with the professional. However, the reasons for their visits could not be established.

Intervention Adherence

Log data were retrieved from the intervention platform. Given the sequential nature of the intervention, the last completed main module was taken as an indication of intervention progress. Furthermore, the number of diary entries and completed elective modules was retrieved.

Reporting and Statistical Analysis

Recruitment data were used to target the first aim of this study. Where this was traceable (eg, through web statistics or information from the participants), the number of potential participants reached and the number of registrations were reported per recruitment method. Subsequent progress, such as returned informed consent forms and final inclusions, was described. Study dropout was reported per condition and assessment point after baseline by calculating the percentage of included participants who did not complete the interviews and self-report questionnaires. Descriptive statistics for all assessed variables were calculated. Intervention and health care use were reported to target the second aim of this report. This comprises the average number of completed sessions, the percentage of elective modules chosen over the entirety of the sessions (as elective modules could be completed multiple times across sessions), and diary entries, as well as the number of students who visited a health care professional and their contact frequency. For the individually guided intervention condition, the median and average number of messages sent between coaches and participants was calculated.

Finally, to tackle the third aim, we carried out a basic analysis of intervention effectiveness on the intention-to-treat sample under a Bayesian framework. If not otherwise specified, all statistical analyses were run in R (R Foundation for Statistical Computing) and RStudio (version 4.1.1; RStudio, Inc [61]; references for auxiliary packages used are provided at the end of this report). A Bayesian linear mixed model using BRMS (version 2.17.0; Bürkner et al [62]) and RStan (version 2.21.5; Guo et al [63]), including both random and fixed effects (chains=4, iterations=4000, and burn-in phase=1000), was conducted separately for the 2 primary outcomes, the QIDS-CR and SIGH-A, whereby all 4 assessment points were nested within individuals. Gender was entered as a covariate. The analyses were repeated for the self-reported measures of depression (PHQ-9) and anxiety (GAD-7), which contained an additional assessment point (midway). The Bayesian approach—in addition to other advantages (see the articles by Wagenmakers et al [64] and by Verhagen and Wagenmakers [65] for a summary)—allowed us to still provide information on the distribution of the small data set without the power restrictions inherent in the frequentist framework. However, to increase the explanatory value of the data, participants from both intervention groups (ie, individually guided and automatically guided) were pooled to constitute 1 intervention condition, which was then compared with CAU in the analyses through an interaction term with assessment point (for a discussion on the results of a sensitivity analysis including all 3 groups separately, see Multimedia Appendix 4). At each assessment point, the hypothesis that the interaction effect (ie, an intervention effect) was different from zero was tested using contrasts comparing each assessment point with the baseline. Moreover, no prior information was included in the analysis, resulting in flat prior distributions [66]. The probability of the collected data emerging under the null model (ie, meaning that the data were a collection of random noise) was compared with the alternative model (ie, condition [intervention vs CAU] had an effect on outcome). The resulting conditional probability was the Bayes factor (BF), which was reported as a measure of the size of the effect for each postbaseline assessment point. Both notations of the BF are reported: BF10, indicating support for the alternative over the null hypothesis, and BF01, indicating support for the null over the alternative hypothesis [67]. When considering the strength of the support, a BF10 of approximately 1 indicates no support, and a BF10 of >10 indicates strong support [68]. High-density intervals were calculated to accompany credibility intervals (CIs) as a measure of the uncertainty of the estimated parameters. Finally, the results of the MINI were used to report the number of individuals who met the diagnostic criteria for a mood or anxiety disorder at the 2 follow-up assessments.

Ethics Approval

The ICare Prevent trial was registered in the International Clinical Trial Registry Platform (ICare Prevent NTR6562) and approved by the medical ethics committees of the Amsterdam University Medical Center (NL6075.029.17 and A2018.166). All participants had to provide informed consent upon registration for the study. A data-sharing agreement between the 2 universities involved in this report was set up. This report has been compiled in accordance with the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) guidelines [69].

Recruitment and Screening

The reach of the recruitment activities could only be determined with certainty for targeted social media advertisements (ie, Facebook and Instagram), defined as the number of individuals who were presented with the advertisement on their computer or smartphone. Reach for other channels such as printed advertisements and sports events was estimated based on publicly available information (eg, number of students and number of attendees). Table 1 provides an overview of the reach and number of registrations per recruitment method, with most registrations stemming from the participant platform (127/202, 62.9%).

Of the 167 students who registered for participation outside of the project collaboration (UvAcare), only 23 (13.8%) returned the signed informed consent form and were assessed for eligibility. Of these 23 individuals, 18 (78%) were excluded at screening based on elevated psychopathology on the MINI (mood disorder: n=7, 39%; anxiety: n=4, 22%; combined mood and anxiety disorder: n=5, 28%; suicide risk: n=2, 11%), 1 (6%) was excluded for being on a waitlist for psychotherapy, and another (6%) was excluded because complaints were too light. This left 13% (3/23) of inclusions, one of which completed the baseline assessment and was randomized. However, this participant dropped out of the study before the midintervention assessment and, thus, only provided baseline data (not reported in this manuscript). Finally, data from 35 students were provided through the collaboration with the UvAcare student mental health project. These students were recruited through their email addresses and provided digital informed consent, making this the most successful approach to recruitment.

Table 1. Reach of potential participants and subsequent registrations for participation per recruitment method (N=202).
Recruitment methodReach, NRegistrations, n (%)
Social media (Instagram and Facebook)122,044a9 (4.5)
Printed advertisements (flyers, posters, and newspapers)115,000b6 (3)
Classrooms and web-based education platformsUnknown3 (1.5)
Participant platformUnknown127 (62.9)
Conference presentationsUnknownUnknown
Running event11,000cUnknown
Targeted emails (UvAcare)Unknown35 (17.3)
Unknown or could not be determinedUnknown22 (10.9)

aActual number based on data extracted from the Facebook advertisement platform.

bEstimate based on official numbers of registered students at Utrecht University, Leiden University, University of Groningen, and University of Amsterdam (2018).

cEstimate based on the number of participants according to the official website.

Participants and Study Dropout

The 35 participants (individually guided intervention: n=14, 40%; automatically guided intervention: n=8, 23%; CAU: n=13, 37%; descriptives in Table 2) had a mean age of 25.86 (SD 4.75) years, with a slight majority of female students (19/35, 54%). A total of 71% (25/35) were undergraduate students, and 29% (10/35) were PhD students.

Study dropout was 57% (20/35; individually guided condition: 8/14, 57%; automatically guided condition: 4/8, 50%; CAU: 8/13, 62%) at the midway assessment, 29% (10/35; individually guided condition: 0%; automatically guided condition: 3/8, 38%; CAU: 7/13, 54%) at the posttest assessment, and 43% (15/35) at both follow-up assessments (6 months: 5/14, 36% in the individually guided condition, 3/8, 38% in the automatically guided condition, and 7/13, 54% in CAU; 12 months: 5/14, 36% in the individually guided condition, 4/8, 50% in the automatically guided condition, and 6/13, 46% in CAU). It is noteworthy that the relatively higher dropout rate at the midassessment was likely related to the fact that it only consisted of a self-report questionnaire and no telephone interview.

Table 2. Descriptive statistics of clinical variables per condition and assessment point.
Time pointQIDS-CRa, mean (SD)Participants, n (%)PHQ-9b, mean (SD)Participants, n (%)SIGH-Ac, mean (SD)Participants, n (%)GAD-7d, mean (SD)Participants, n (%)
Total sample (N=35)e

Baseline5.51 (2.84)35 (100)7.54 (3.79)35 (100)5.77 (4.51)35 (100)6.23 (3.72)35 (100)

Midwayfg5.13 (2.5)15 (43)4.47 (3.7)15 (43)

Posttest assessmenth4.92 (2.77)25 (71)6.0 (5.26)7 (20)5.32 (4.95)25 (71)4.43 (4.24)7 (20)

6-month follow-up4.92 (2.53)13 (37)5.58 (4.56)19 (54)3.69 (4.29)13 (37)4.63 (3.72)19 (54)

12-month follow-up5.13 (3.59)16 (46)5.25 (3.64)20 (57)1.06 (1.44)16 (46)4.2 (3.82)20 (57)
Individually guided condition (n=14)i

Baseline4.79 (2.61)14 (100)6.14 (2.03)14 (100)4.71 (3.36)14 (100)4.93 (3.08)14 (100)

Midway4.0 (2.9)6 (43)3.17 (2.71)6 (43)

Posttest assessment4.86 (3.23)14 (100)3.0 (1.83)4 (29)5.57 (5.92)14 (100)2.0 (1.83)4 (29)

6-month follow-up5.0 (3.37)4 (29)4.22 (2.64)9 (64)1.75 (2.06)4 (29)3.22 (3.11)9 (64)

12-month follow-up4.14 (2.73)7 (50)4.78 (3.56)9 (64)1.0 (1.53)7 (50)4.11 (3.79)9 (64)
Automatically guided condition (n=8)j

Baseline5.88 (2.64)8 (100)8.0 (3.82)8 (100)6.0 (5.61)8 (100)7.5 (4.93)8 (100)

Midway6.25 (3.2)4 (50)7.0 (5.89)4 (50)

Posttest assessment4.2 (2.05)5 (62)N/Ak0 (0)5.0 (4.36)5 (62)N/A0 (0)

6-month follow-up3.8 (1.92)5 (62)2.75 (3.59)4 (50)2.8 (4.38)5 (62)3.5 (5.2)4 (50)

12-month follow-up2.0 (0.0)2 (25)2.25 (1.71)4 (50)0.0 (0.0)2 (25)2.25 (4.5)4 (50)
CAUl condition (n=13)m

Baseline6.1 (3.23)13 (100)8.77 (4.87)13 (100)6.77 (4.97)13 (100)6.85 (3.36)13 (100)

Midway5.6 (0.55)5 (38)4.0 (1.87)5 (38)

Posttest assessment5.68 (2.25)6 (46)10.0 (6.0)3 (23)5.0 (3.29)6 (46)7.67 (4.62)3 (23)

6-month follow-up6.25 (2.22)4 (31)9.5 (5.28)6 (46)6.75 (4.99)4 (31)7.5 (1.87)6 (46)

12-month follow-up7.0 (4.04)7 (54)7.57 (3.31)7 (54)1.43 (1.51)7 (54)5.43 (3.55)7 (54)

aQIDS-CR: Quick Inventory of Depressive Symptomatology-Clinician Rated [48].

bPHQ-9: Patient Health Questionnaire [59].

cSIGH-A: Structured Interview Guide for the Hamilton Anxiety Rating Scale [53,54].

dGAD-7: Generalized Anxiety Disorder Scale [42].

eMean age 25.86 (SD 4.75) years; 54% (19/35) women; 46% (16/35) men.

fMidway: 5 weeks after randomization; only self-report assessed.

gNot available.

hPosttest assessment: 8 weeks after randomization.

iMean age 27.86 (SD 6.67) years; 64% (9/14) women; 36% (5/14) men.

jMean age 25.5 (SD 1.41) years; 50% (4/8) women; 50% (4/8) men.

kN/A: not applicable.

lCAU: care as usual.

mMean age 23.92 (SD 2.4) years; 46% (6/13) women; 54% (7/13) men.

Intervention and Health Care Use

The 63% (22/35) of participants in the intervention conditions completed an average of 3 sessions (SD 2.9; individually guided condition: mean 4, SD 2.9; automatically guided condition: mean 2, SD 2.4). A total of 7 participants (individually guided condition: n=3, 43%; automatically guided condition: n=4, 57%) started but did not complete the first session, and only 7% (1/14) of the participants in the individually guided condition completed the booster module. In most sessions that offered the selection of an elective module (ie, sessions 2 to 7; 19/49, 39%), no elective module was chosen. Across all these sessions, the most frequently selected module dealt with improving self-esteem (9/49, 18%), followed by gratitude (6/49, 12%), improving sleep, and dealing with perfectionism (5/49, 10% each). The elective module teaching acceptance techniques was chosen in 6% (3/49) of all completed sessions. The relaxation, rumination, and reducing alcohol consumption modules were chosen the least frequently, namely, in 2% (1/49) of all completed sessions each. Only 14% (2/14) of the participants in the individually guided condition used the diary function, with a total of 8 entries in the activity diary. On average, the coaches and participants in this condition exchanged 5 (SD 4.79; median 3) messages.

At posttest assessment, 4 students (individually guided condition: n=2, 50%; CAU: n=2, 50%) indicated that they had consulted one or more health care professionals in the previous 2 months (GP: n=3, 75%; student psychologist or psychotherapist: n=2, 50%; study advisor or counselor: n=2, 50%; other: n=1, 25%). Frequencies ranged from 1 to 3 visits. This increased to 11 students (individually guided condition: n=4, 36%; automatically guided condition: n=3, 27%; CAU: n=4, 36%) at the 6-month follow-up (GP: n=4, 36%; study advisor or counselor: n=4, 36%; student psychologist or psychotherapist: n=4, 36%; self-help group: n=1, 9%; other: n=4, 36%), with frequencies ranging from single to weekly visits (self-help group). At the 12-month follow-up, 10 students (individually guided condition: n=4, 40%; automatically guided condition: n=2, 20%; CAU: n=4, 40%) had visited one or more health care professionals (study advisor or counselor: n=1, 10%; student psychologist or psychotherapist: n=5, 50%; other: n=3, 30%). The visits ranged from 1 to 7 times.

Symptom Change and Diagnoses

The results on symptom change (Table 3) suggest moderate to strong support for the null hypothesis (ie, the absence of an intervention effect) on almost all mental health outcomes at all assessment points (BF10 range=0.03-2.7; BF01 range=0.37-35.36) [68]. Values of >1 for BF10, which indicates support for the hypothesis that an intervention effect is present, were found exclusively in measures of anxiety. Specifically, for the GAD-7, the interaction between condition and time (β=−0.39, 95% CI −3.78 to 2.91; BF01=0.7) was 1.43 times more likely to emerge under the alternative hypothesis than under the null hypothesis at the midway assessment compared with the baseline. For the SIGH-A, this interaction was found to be 1.64 times more likely at posttest assessment versus baseline (β=−0.82, 95% CI −6.18 to 4.52; BF01=0.61) and 2.7 times more likely at the 12-month follow-up versus baseline (β=−1.36, 95% CI −5.86 to 3.13; BF01=0.37) under the alternative hypothesis.

Although these effects indicate a superior effect of CAU over the intervention condition, as CAU shows stronger reductions in symptoms compared with baseline at these assessment points, the strength of the support is only anecdotal [68] and is associated with significant uncertainty, as indicated by both the CIs and high-density intervals, reflecting the small sample size (see also Figures 2-5). Thus, there is no reliable evidence of an intervention effect (Table 3).

At the 6-month follow-up assessment, 13 students completed the telephone interviews, of whom 1 (8%) in the individually guided intervention condition presented with a GAD according to the MINI. In total, 16 students completed the 12-month follow-up interviews, of whom 2 (12%) in the CAU condition met diagnostic criteria: 1 (50%) for dysthymia and another (50%) for GAD. Another 14% (2/14) of the students in the individually guided condition presented with mixed anxiety and depression.

Table 3. Estimates and Bayes factors (BFs) for the interaction with condition per assessment point and instrument.
Time pointEstimate, β95% CIa95% HDIbBF10cBF01d

Baseline vs posttest assessment1.25−1.63 to 4.09−1.62 to

Baseline vs 6-month follow-up2.29−1.19 to 5.79−1.00 to 5.960.1010.18

Baseline vs 12-month follow-up2.21−0.65 to 5.04−0.58 to 5.080.0615.67

Baseline vs midway.34−4.47 to 5.13−4.65 to 4.930.781.28

Baseline vs posttest assessment3.36−5.12 to 11.6−5.42 to

Baseline vs 6-month follow-up4.10−1.24 to 9.44−1.43 to 9.210.0615.78

Baseline vs 12-month follow-up1.70−2.21 to 5.60−2.10 to 5.700.234.30

Baseline vs posttest assessment−.82−6.18 to 4.52−6.13 to 4.541.640.61

Baseline vs 6-month follow-up3.94−2.15 to 10.18−2.39 to 9.900.109.88

Baseline vs 12-month follow-up−1.36−5.86 to 3.13−5.92 to 3.072.700.37

Baseline vs midway−.39−3.78 to 2.91−3.78 to 2.911.430.70

Baseline vs posttest assessment3.07−3.53 to 9.64−3.79 to

Baseline vs 6-month follow-up3.23−0.09 to 6.530.09 to 6.690.0335.36

Baseline vs 12-month follow-up.68−3.07 to 4.60−3.09 to 4.570.561.79

aCI: credibility interval.

bHDI: high-density interval.

cBF10: BF indicating probability of alternative over null hypothesis.

dBF01: BF indicating probability of null over alternative hypothesis.

eQIDS-CR: Quick Inventory of Depressive Symptomatology-Clinician Rated [48].

fPHQ-9: Patient Health Questionnaire [59].

gSIGH-A: Structured Interview Guide for the Hamilton Anxiety Rating Scale.

hGAD-7: Generalized Anxiety Disorder Scale [42].

Figure 2. Depression scores (Quick Inventory of Depressive Symptomatology-Clinician Rated [QIDS-CR]) per assessment point and pooled condition with 95% credibility intervals.
View this figure
Figure 3. Self-reported depression scores (Patient Health Questionnaire [PHQ-9]) per assessment point and pooled condition with 95% credibility intervals.
View this figure
Figure 4. Anxiety scores (Structured Interview Guide for the Hamilton Anxiety Rating Scale [SIGH-A]) per assessment point and pooled condition with 95% credibility intervals.
View this figure
Figure 5. Self-reported anxiety scores (Generalized Anxiety Disorder Scale [GAD-7]) per assessment point and pooled condition with 95% credibility intervals.
View this figure

Despite the availability of effective low-threshold mental health interventions, such as those provided via the internet [28], students seem to be difficult to recruit for interventions on this topic. Therefore, the aims of this study were to (1) describe the recruitment process for an RCT of a web-based intervention for the prevention of depression and anxiety in students, (2) describe how participants used the intervention, and (3) conduct a basic analysis of intervention effects.

Recruitment Process

We used an extensive and multifaceted recruitment strategy, ranging from traditional print and social media advertisements to more unconventional means such as participation in a sports event. Ultimately, direct contact through students’ email addresses was the most effective recruitment channel, which was a unique opportunity in the UvAcare project [44]. A larger number of students could be reached efficiently at the same time, although this method focused largely on screening students at the beginning of a semester, when exam anxiety and other stressors might not have set in yet. It is possible that more students would have fulfilled the inclusion criteria toward the end of the semester. However, this strategy potentially reduced the feeling of being individually targeted for mental health complaints. Although, to the best of our knowledge, this has not been tested yet, it is conceivable that—compared with a universal recruitment strategy—an email from the central student administration may have led to more compliant behavior in completing the screener and providing informed consent for the subsequent RCT. Moreover, the use of digital informed consent reduced the barrier to participation in the UvAcare project. Printing and posting a form, as required in the initial trial, likely negated the benefits of a digital intervention (eg, its ease of access). A recently conducted RCT supports this notion [70]. The authors assessed the effectiveness of the ICare Prevent intervention on symptom reduction rather than its preventive potential. Although their inclusion and exclusion criteria were less restrictive than those reported in this study and the target sample size was eventually reached, ≤3% of all recruited students could be randomized, largely because students did not provide written informed consent. It needs to be stated that the COVID-19 pandemic has led the evaluating Medical Ethics Committee to reconsider its reluctance to allow digital informed consent. In addition, the possibility that students genuinely did not want to consent to the research procedure cannot be excluded.

In line with the aforementioned potential explanations, the individual approach through the participant platform generated by far the largest number of study sign-ups. As the platform was geared toward participation in clinical studies, students who had registered were likely already aware of health complaints, either physical or mental, both of which are often comorbid [71]. Consequently, almost all students recruited through this channel exceeded the clinical cutoff for mental health problems. It is noteworthy that the screening instrument we used (MINI) has been shown to be overly sensitive and, therefore, likely overestimated the prevalence of major depression in our sample [58]. However, the high degree of psychopathology also sheds light on another potential underlying issue, namely, students’ attitudes toward help seeking in general [31]. This problem might also explain why the most efficient approach to recruitment (ie, direct contact through email addresses) resulted in the randomization of only 35 students.

Although, in recent mental health surveys among students in the Netherlands, a response rate of <12% was found [72], detailed accounts of recruitment challenges are generally limited to treatment studies in nonstudent populations. An early systematic review of 78 studies summarized a decade of barriers to trial recruitment [73]. In line with our findings, the most common concerns were related to the information and consent procedure and the inconvenience caused by the research context (eg, strict procedures, additional appointments, and travel time, the latter being less applicable as there is no travel time and limited in-person contact with physicians or scientific staff in trials on digital interventions). Subsequent reviews have confirmed the relevance of these concerns to recruitment and participation [74,75]. In an attempt to quantify the problem, a study systematically reviewed 1017 study protocols of RCTs, most from the field of oncology and cardiovascular medicine, and compared their available recruitment and publication status [76]. Approximately one-quarter of the studies were terminated prematurely, most commonly because of recruitment challenges and, of these, only 40% were published [76]. These challenges are likely more pronounced in prevention studies [29] considering that individuals at prodromal stages of, for example, depression might not feel the urge to seek help and might prefer self-management of complaints. On the basis of these findings, a set of general recommendations [30,77] should be emphasized.

First, we encourage complementing RCTs of new interventions or in new target groups with well-designed feasibility trials [77]. A recent example is provided by a study assessing the feasibility and acceptability of the ICare Prevent intervention in Indonesian students, with positive results [78]. Importantly, such web-based interventions are novel in Indonesia, and treatment for mental health complaints is limited in the country, potentially explaining why recruitment was less problematic. Conversely, another study reported substantial challenges in recruiting participants, with the informed consent procedure again being a major barrier [79]. In this feasibility study, the ICare Prevent intervention was adapted for patients with coronary artery disease. Although such feasibility or pilot trials are regularly conducted, it is advisable to routinely extend the assessment of the feasibility, acceptability, and cost-effectiveness to the potential recruitment measures and report these in outcome papers. Therefore, a structured and preplanned evaluation of recruitment strategies is important. For example, a study indicated that recruitment via posters and websites was most cost-effective in the context of an RCT on depression relapse prevention while noting that a multifaceted recruitment strategy was crucial [80]. In our study, such a diverse set of strategies proved unsuccessful. In line with this, a review of effective approaches to recruiting participants for RCTs has shown that the quality of available evidence is low and that those effective approaches that emerged from higher-quality studies, such as unblinding trials and calling potential participants, are not applicable in many research contexts [81]. On a brighter note, the dearth of information has advanced initiatives such as the Prioritising Recruitment in Randomised Trials project [82], which entails a web-based compilation of important recruitment-related questions and answers provided by experts. It offers an open-access database of expert-based suggestions on ways to improve recruitment for clinical trials.

Second, cocreation with end users should be considered beyond intervention development. Notably, a systematic review suggested that involving participants in the creation of study information material was not efficient [81]. However, active participation of students in the identification of effective recruitment channels beyond development (eg, flyers and posters) would allow for a targeted recruitment strategy. In line with this idea, although a broad recruitment strategy was deemed most effective initially for the ICare Prevent trial, the results outlined previously show that saving efforts for specific channels is more efficient.

Finally, it has been suggested to overcome the relatively low uptake of preventive mental health interventions by focusing on indirect prevention—interventions that do not directly address mental health problems but rather target related issues (eg, insomnia) with the aim of positively affecting mental health conditions [30]. More research is needed to investigate this possibility, but examples of digital interventions that use such an indirect prevention approach exist, for example, a set of Complaint-Directed Mini-Interventions for depression, evaluated in the general population [83]. These interventions consist of short web-based modules for improving sleep, reducing stress, and tackling excessive worrying. The authors found not only a significant reduction in depressive symptoms at the 3-month follow-up compared with a waitlist control (d=−0.7) but also a promising sign-up rate and provision of informed consent. In the context of our study, the prominent focus on depression and anxiety might have deterred students at subclinical stages. As noted previously, these students might not have identified as having depressive or anxiety complaints, thereby reducing the need for seeking help. Moreover, in the absence of an urge to seek help, the research context (eg, informed consent, interviews, and questionnaires) might have been experienced as too burdensome. This serves as a potential explanation for why mostly those students with clinical presentations of mental health conditions signed up for the RCT.

Intervention Use and Effects

The use of the intervention was low, with an average of 3 sessions completed by the participants. Some of the main intervention components were presented in these first sessions, such as psychoeducation and behavioral activation. However, the transdiagnostic components, which were mainly contained in the elective modules, were only provided from the second session onward, and other core elements such as cognitive restructuring followed in later sessions. Moreover, the analyses did not suggest an intervention effect on depression or anxiety outcomes. Only anecdotal evidence for the added benefit of individual guidance was obtained as the only participant who completed the booster session received support from a coach. Whether the low use of the intervention also affected the absence of intervention effects on mental health outcomes cannot be established because of the small sample size. However, these findings complement studies on the same intervention conducted in different contexts. Although one study reported positive within-group effects immediately after the intervention (depression: d=0.42; anxiety: d=1.19) [84], another study found no difference between the ICare Prevent intervention and CAU in an RCT among students [70]. Moreover, a meta-analysis on digital mental health interventions in students has suggested that the effect sizes of such interventions are small at best in this population (depression: d=0.18; anxiety: d=0.27), and subsequent trials will likely include null findings (prediction interval for depression: −0.26 to 0.62; prediction interval for anxiety: −0.36 to 0.90) [28]. Therefore, it is conceivable that the ICare Prevent intervention requires further adaptation before being suitable for students.

Limitations and Strengths

This study has a number of limitations. The encountered recruitment challenges resulted in a small sample size, which had considerable implications. First, it meant that we could not follow the analytical approach outlined in the study protocol [40]. Additional analyses (eg, on the relationship between dropout and outcome) were not possible. Moreover, pooling the individually and automatically guided intervention conditions was a compromise to reach stable statistical models that would allow for a description of the data distribution (see also Multimedia Appendix 4). Previous research has consistently indicated the superiority of individually guided interventions, specifically those involving human rather than technical support [85], although this applies less so for subclinical stages of depression [86]. Therefore, any suggestions and findings need to be considered exploratory and incidental and require replication. However, we chose the alternative Bayesian framework to overcome the sample size limitations inherent in frequentist statistics and provide a basic account of the intervention’s effectiveness. Although we used noninformative priors, a strength of this study is that it provides a starting point for future trials. Essentially, subsequent RCTs on the topic of transdiagnostic preventive digital health interventions for students can build on our data and use them as prior information [64,87]. This would allow for a continuous research circle from feasibility trial, which generates the prior information, to full RCT, which provides up-to-date information.

Moreover, we did not use a structured evaluation of the recruitment challenges or a broader process evaluation as this was not part of the originally planned RCT. This includes a systematic appraisal of recruitment costs, which we have not reported on because of a lack of reliable data. Although this is a lesson learned for future RCTs, reporting both unforeseen recruitment challenges and data on participants and their use of the intervention is important nevertheless. For example, evidence of the effectiveness of interventions might suffer from publication bias. In this regard, a strength of this study is the publication of data that can be beneficial not only in guiding future studies on developing recruitment strategies but also in data synthesis efforts such as meta-analyses. The high costs involved in clinical trials and the intensive involvement of both staff and participants mandate such reporting of the collected data, which hopefully can aid in preventing similar situations in the future and, therefore, save scarce resources [76].

Conclusions and Future Research

The recruitment of students for digital mental health interventions that focus on the prevention of depression and anxiety is difficult. Although targeted approaches such as direct email contact seem the most efficient, more research is needed on factors that can improve recruitment, and subsequent improved strategies need to be developed. Moreover, evidence on whether direct rather than indirect prevention efforts are suitable for this target group is mixed and requires further investigation. We provided an account of recruitment challenges as well as basic information on intervention effects that can aid future studies in the development and evaluation of similar interventions.


The ICare Prevent trial was funded by the European Commission Horizon 2020 Research and Innovation action framework under grant agreement 634757. The authors would like to thank Marleen Swenne for her work on adapting the intervention to the student context; Angelika Stefan for the initial brainstorming about Bayesian statistics; Jaël Sierra van Bentum for implementing the randomization sequence; Nine Wolters, Ruben Atteveld, and Anke Klein for their work on the UvAcare trial; Vincent Palte and Fleur Lock for conducting the clinical interviews; and Joost Gladpootjes, Hinna Mirza, Nurdane Aslan, and Khadicha Amarti for recruitment.

Data Availability

The statistical packages in RStudio (RStudio, Inc) used were as follows: bayestestR (version 0.12.1 [88]), foreign (version 0.8-82 [89]), ggplot2 (version 3.3.5 [90]), Hmisc (version 4.7-0. [91]), and Psych (version 2.2.3. [92]). Full analytical code is available [93].

Authors' Contributions

FB contributed to conceptualization, methodology, formal analysis, investigation, data curation, writing (original draft), visualization, and project administration. AK contributed to conceptualization, investigation, writing (review and editing), supervision, and funding acquisition. KN contributed to methodology, data curation, and writing (review and editing). EK contributed to supervision and writing (review and editing). RW contributed to supervision and writing (review and editing). LdK contributed to data curation and writing (review and editing). CJ contributed to conceptualization, writing (review and editing), and funding acquisition. ACZ contributed to software and writing (review and editing). KKW contributed to software and writing (review and editing). PC contributed to supervision and writing (review and editing). HR contributed to conceptualization, investigation, writing (review and editing), supervision, and funding acquisition. All authors have read and approved the final manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Recruitment poster.

PNG File , 2826 KB

Multimedia Appendix 2

Recruitment sticker.

PNG File , 864 KB

Multimedia Appendix 3

Intervention description.

DOCX File , 13 KB

Multimedia Appendix 4

Sensitivity analysis on all conditions.

DOCX File , 48 KB

Multimedia Appendix 5

CONSORT-eHEALTH checklist (V 1.6.1).

PDF File (Adobe PDF File), 1185 KB

  1. Kalin NH. The critical relationship between anxiety and depression. Am J Psychiatry 2020 May 01;177(5):365-367. [CrossRef] [Medline]
  2. Lamers F, van Oppen P, Comijs HC, Smit JH, Spinhoven P, van Balkom AJ, et al. Comorbidity patterns of anxiety and depressive disorders in a large cohort study: the Netherlands Study of Depression and Anxiety (NESDA). J Clin Psychiatry 2011 Mar;72(3):341-348. [CrossRef] [Medline]
  3. Lijster JM, Dierckx B, Utens EM, Verhulst FC, Zieldorff C, Dieleman GC, et al. The age of onset of anxiety disorders. Can J Psychiatry 2017 Apr;62(4):237-246 [FREE Full text] [CrossRef] [Medline]
  4. Solmi M, Radua J, Olivola M, Croce E, Soardo L, Salazar de Pablo G, et al. Age at onset of mental disorders worldwide: large-scale meta-analysis of 192 epidemiological studies. Mol Psychiatry 2022 Jan;27(1):281-295 [FREE Full text] [CrossRef] [Medline]
  5. Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, WHO WMH-ICS Collaborators. WHO World Mental Health Surveys International College Student Project: prevalence and distribution of mental disorders. J Abnorm Psychol 2018 Oct;127(7):623-638 [FREE Full text] [CrossRef] [Medline]
  6. de Girolamo G, McGorry PD, Sartorius N. Age of Onset of Mental Disorders: Etiopathogenetic and Treatment Implications. Berlin, Germany: Springer; 2019.
  7. Newcomb-Anjo SE, Villemaire-Krajden R, Takefman K, Barker ET. The unique associations of academic experiences with depressive symptoms in emerging adulthood. Emerg Adulthood 2016 Jul 07;5(1):75-80. [CrossRef]
  8. Barker ET, Howard AL, Villemaire-Krajden R, Galambos NL. The rise and fall of depressive symptoms and academic stress in two samples of university students. J Youth Adolesc 2018 Jun;47(6):1252-1266. [CrossRef] [Medline]
  9. Karyotaki E, Cuijpers P, Albor Y, Alonso J, Auerbach RP, Bantjes J, et al. Sources of stress and their associations with mental disorders among college students: results of the World Health Organization world mental health surveys international college student initiative. Front Psychol 2020 Jul 30;11:1759 [FREE Full text] [CrossRef] [Medline]
  10. Beiter R, Nash R, McCrady M, Rhoades D, Linscomb M, Clarahan M, et al. The prevalence and correlates of depression, anxiety, and stress in a sample of college students. J Affect Disord 2015 Mar 01;173:90-96. [CrossRef] [Medline]
  11. Savage MJ, James R, Magistro D, Donaldson J, Healy LC, Nevill M, et al. Mental health and movement behaviour during the COVID-19 pandemic in UK university students: prospective cohort study. Ment Health Phys Act 2020 Oct;19:100357 [FREE Full text] [CrossRef]
  12. Odriozola-González P, Planchuelo-Gómez Á, Irurtia MJ, de Luis-García R. Psychological effects of the COVID-19 outbreak and lockdown among students and workers of a Spanish university. Psychiatry Res 2020 Aug;290:113108 [FREE Full text] [CrossRef] [Medline]
  13. Robb CA. College student financial stress: are the kids alright? J Fam Econ Iss 2017 Apr 19;38(4):514-527. [CrossRef]
  14. Richardson T, Elliott P, Roberts R, Jansen M. A longitudinal study of financial difficulties and mental health in a national sample of British undergraduate students. Community Ment Health J 2017 Apr;53(3):344-352 [FREE Full text] [CrossRef] [Medline]
  15. Cuijpers P, Auerbach RP, Benjet C, Bruffaerts R, Ebert D, Karyotaki E, et al. The World Health Organization world mental health international college student initiative: an overview. Int J Methods Psychiatr Res 2019 Jun;28(2):e1761 [FREE Full text] [CrossRef] [Medline]
  16. Jenkins PE, Ducker I, Gooding R, James M, Rutter-Eley E. Anxiety and depression in a sample of UK college students: a study of prevalence, comorbidity, and quality of life. J Am Coll Health 2021;69(8):813-819. [CrossRef] [Medline]
  17. Alonso J, Mortier P, Auerbach RP, Bruffaerts R, Vilagut G, Cuijpers P, WHO WMH-ICS Collaborators. Severe role impairment associated with mental disorders: results of the WHO World Mental Health Surveys International College Student Project. Depress Anxiety 2018 Sep;35(9):802-814 [FREE Full text] [CrossRef] [Medline]
  18. Mortier P, Demyttenaere K, Auerbach RP, Green JG, Kessler RC, Kiekens G, et al. The impact of lifetime suicidality on academic performance in college freshmen. J Affect Disord 2015 Nov 01;186:254-260 [FREE Full text] [CrossRef] [Medline]
  19. Bruffaerts R, Mortier P, Kiekens G, Auerbach RP, Cuijpers P, Demyttenaere K, et al. Mental health problems in college freshmen: prevalence and academic functioning. J Affect Disord 2018 Jan 01;225:97-103 [FREE Full text] [CrossRef] [Medline]
  20. Bodden DH, van den Heuvel MW, Engels RC, Dirksen CD. Societal costs of subclinical depressive symptoms in Dutch adolescents: a cost-of-illness study. J Child Psychol Psychiatry 2022 Jul;63(7):771-780 [FREE Full text] [CrossRef] [Medline]
  21. Agasisti T, Bertoletti A. Higher education and economic growth: a longitudinal study of European regions 2000–2017. Socio-Econ Plan Sci 2022 Jun;81:100940 [FREE Full text] [CrossRef]
  22. Institute of Medicine (US) Committee on Prevention of Mental Disorders. In: Mrazek PJ, Haggerty RJ, editors. Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research. Washington, DC, USA: National Academies Press (US); 1994.
  23. Buntrock C, Ebert DD, Lehr D, Smit F, Riper H, Berking M, et al. Effect of a web-based guided self-help intervention for prevention of major depression in adults with subthreshold depression: a randomized clinical trial. JAMA 2016 May 03;315(17):1854-1863. [CrossRef] [Medline]
  24. McDaid D, Park AL, Wahlbeck K. The economic case for the prevention of mental illness. Annu Rev Public Health 2019 Apr 01;40:373-389. [CrossRef] [Medline]
  25. Cuijpers P, Pineda BS, Quero S, Karyotaki E, Struijs SY, Figueroa CA, et al. Psychological interventions to prevent the onset of depressive disorders: a meta-analysis of randomized controlled trials. Clin Psychol Rev 2021 Mar;83:101955 [FREE Full text] [CrossRef] [Medline]
  26. Moreno-Peral P, Conejo-Cerón S, Rubio-Valera M, Fernández A, Navas-Campaña D, Rodríguez-Morejón A, et al. Effectiveness of psychological and/or educational interventions in the prevention of anxiety: a systematic review, meta-analysis, and meta-regression. JAMA Psychiatry 2017 Oct 01;74(10):1021-1029 [FREE Full text] [CrossRef] [Medline]
  27. Barnett P, Arundell LL, Saunders R, Matthews H, Pilling S. The efficacy of psychological interventions for the prevention and treatment of mental health disorders in university students: a systematic review and meta-analysis. J Affect Disord 2021 Mar 01;280(Pt A):381-406. [CrossRef] [Medline]
  28. Harrer M, Adam SH, Baumeister H, Cuijpers P, Karyotaki E, Auerbach RP, et al. Internet interventions for mental health in university students: a systematic review and meta-analysis. Int J Methods Psychiatr Res 2019 Jun;28(2):e1759 [FREE Full text] [CrossRef] [Medline]
  29. Cuijpers P, van Straten A, Warmerdam L, van Rooy MJ. Recruiting participants for interventions to prevent the onset of depressive disorders: possible ways to increase participation rates. BMC Health Serv Res 2010 Jun 25;10:181 [FREE Full text] [CrossRef] [Medline]
  30. Cuijpers P. Indirect prevention and treatment of depression: an emerging paradigm? Clin Psychol Eur 2021 Dec;3(4):e6847 [FREE Full text] [CrossRef] [Medline]
  31. Ebert DD, Mortier P, Kaehlke F, Bruffaerts R, Baumeister H, Auerbach RP, WHO World Mental Health-International College Student Initiative collaborators. Barriers of mental health treatment utilization among first-year college students: first cross-national results from the WHO World Mental Health International College Student Initiative. Int J Methods Psychiatr Res 2019 Jun;28(2):e1782 [FREE Full text] [CrossRef] [Medline]
  32. Hirsch JK, Rabon JK, Reynolds EE, Barton AL, Chang EC. Perceived stress and suicidal behaviors in college students: conditional indirect effects of depressive symptoms and mental health stigma. Stigma Health 2019 Feb;4(1):98-106. [CrossRef]
  33. Wada M, Suto MJ, Lee M, Sanders D, Sun C, Le TN, et al. University students’ perspectives on mental illness stigma. Ment Health Prev 2019 Jun;14:200159 [FREE Full text] [CrossRef]
  34. Xiao H, Carney DM, Youn SJ, Janis RA, Castonguay LG, Hayes JA, et al. Are we in crisis? National mental health and treatment trends in college counseling centers. Psychol Serv 2017 Nov;14(4):407-415. [CrossRef] [Medline]
  35. Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, et al. Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychol Med 2016 Oct;46(14):2955-2970 [FREE Full text] [CrossRef] [Medline]
  36. Ennis E, McLafferty M, Murray E, Lapsley C, Bjourson T, Armour C, et al. Readiness to change and barriers to treatment seeking in college students with a mental disorder. J Affect Disord 2019 Jun 01;252:428-434. [CrossRef] [Medline]
  37. Küchler AM, Schultchen D, Pollatos O, Moshagen M, Ebert DD, Baumeister H. StudiCare mindfulness-study protocol of a randomized controlled trial evaluating an internet- and mobile-based intervention for college students with no and "on demand" guidance. Trials 2020 Nov 26;21(1):975 [FREE Full text] [CrossRef] [Medline]
  38. Păsărelu CR, Andersson G, Bergman Nordgren L, Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn Behav Ther 2017 Jan;46(1):1-28. [CrossRef] [Medline]
  39. Cuijpers P. Why primary prevention often is no prevention at all. Eur Neuropsychopharmacol 2022 May;58:1-3. [CrossRef] [Medline]
  40. Bolinski F, Kleiboer A, Karyotaki E, Bosmans JE, Zarski AC, Weisel KK, et al. Effectiveness of a transdiagnostic individually tailored Internet-based and mobile-supported intervention for the indicated prevention of depression and anxiety (ICare Prevent) in Dutch college students: study protocol for a randomised controlled trial. Trials 2018 Feb 20;19(1):118 [FREE Full text] [CrossRef] [Medline]
  41. Radloff L. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas 1977 Jun;1(3):385-401. [CrossRef]
  42. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
  43. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59 Suppl 20:22-57. [Medline]
  44. Klein A, Wolters NE, Bol EJM, Koelen J, de Koning L, Roetink SS, et al. Online computer or therapist-guided cognitive behavioral therapy in university students with anxiety and/or depression: study protocol of a randomised controlled trial. BMJ Open 2021 Nov 26;11(11):e049554 [FREE Full text] [CrossRef] [Medline]
  45. Link2trials. 2017.   URL: [accessed 2022-12-06]
  46. Results of the 2018 Dam tot Damloop. Le Champion. 2018.   URL: [accessed 2022-12-06]
  47. Weisel KK, Zarski AC, Berger T, Krieger T, Schaub MP, Moser CT, et al. Efficacy and cost-effectiveness of guided and unguided internet- and mobile-based indicated transdiagnostic prevention of depression and anxiety (ICare Prevent): a three-armed randomized controlled trial in four European countries. Internet Interv 2019 Apr;16:52-64 [FREE Full text] [CrossRef] [Medline]
  48. Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, et al. The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry 2003 Sep 01;54(5):573-583. [CrossRef] [Medline]
  49. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). 4th edition. Washington, DC, USA: American Psychiatric Association; 1994.
  50. Trivedi MH, Rush AJ, Ibrahim HM, Carmody TJ, Biggs MM, Suppes T, et al. The Inventory of Depressive Symptomatology, Clinician Rating (IDS-C) and Self-Report (IDS-SR), and the Quick Inventory of Depressive Symptomatology, Clinician Rating (QIDS-C) and Self-Report (QIDS-SR) in public sector patients with mood disorders: a psychometric evaluation. Psychol Med 2004 Jan;34(1):73-82. [CrossRef] [Medline]
  51. Bernstein IH, Rush AJ, Suppes T, Trivedi MH, Woo A, Kyutoku Y, et al. A psychometric evaluation of the clinician-rated Quick Inventory of Depressive Symptomatology (QIDS-C16) in patients with bipolar disorder. Int J Methods Psychiatr Res 2009 Jun;18(2):138-146 [FREE Full text] [CrossRef] [Medline]
  52. Rush AJ, Bernstein IH, Trivedi MH, Carmody TJ, Wisniewski S, Mundt JC, et al. An evaluation of the quick inventory of depressive symptomatology and the hamilton rating scale for depression: a sequenced treatment alternatives to relieve depression trial report. Biol Psychiatry 2006 Mar 15;59(6):493-501 [FREE Full text] [CrossRef] [Medline]
  53. Shear MK, Vander Bilt J, Rucci P, Endicott J, Lydiard B, Otto MW, et al. Reliability and validity of a structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A). Depress Anxiety 2001;13(4):166-178. [Medline]
  54. Hamilton M. The assessment of anxiety states by rating. Br J Med Psychol 1959;32(1):50-55. [CrossRef] [Medline]
  55. Clark DB, Donovan JE. Reliability and validity of the Hamilton Anxiety Rating Scale in an adolescent sample. J Am Acad Child Adolesc Psychiatry 1994;33(3):354-360. [CrossRef] [Medline]
  56. Williams JB, Kobak KA, Giller E, Reasner DS, Curry L, Detke MJ. Comparison of site-based versus central ratings in a study of generalized anxiety disorder. J Clin Psychopharmacol 2015 Dec;35(6):654-660. [CrossRef] [Medline]
  57. Sheehan DV, Sheehan KH, Shytle RD, Janavs J, Bannon Y, Rogers JE, et al. Reliability and validity of the mini international neuropsychiatric interview for children and adolescents (MINI-KID). J Clin Psychiatry 2010 Mar;71(3):313-326. [CrossRef] [Medline]
  58. Levis B, Benedetti A, Riehm KE, Saadat N, Levis AW, Azar M, et al. Probability of major depression diagnostic classification using semi-structured versus fully structured diagnostic interviews. Br J Psychiatry 2018 Jun;212(6):377-385 [FREE Full text] [CrossRef] [Medline]
  59. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
  60. Donker T, van Straten A, Marks I, Cuijpers P. Quick and easy self-rating of Generalized Anxiety Disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res 2011 Jun 30;188(1):58-64. [CrossRef] [Medline]
  61. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2021.   URL: [accessed 2022-12-06]
  62. Bürkner PC, Gabry J, Weber S, Johnson A, Modrak M, Badr HS, et al. brms: Bayesian Regression Models using 'Stan'. V. 2.17.0. The Comprehensive R Archive Network. 2022.   URL: [accessed 2022-12-06]
  63. Guo J, Gabry J, Goodrich B, Weber S. rstan: R Interface to Stan. v. 2.21.7. The Comprehensive R Archive Network. 2022.   URL: [accessed 2022-12-06]
  64. Wagenmakers EJ, Marsman M, Jamil T, Ly A, Verhagen J, Love J, et al. Bayesian inference for psychology. Part I: theoretical advantages and practical ramifications. Psychon Bull Rev 2018 Mar;25(1):35-57 [FREE Full text] [CrossRef] [Medline]
  65. Verhagen J, Wagenmakers EJ. Bayesian tests to quantify the result of a replication attempt. J Exp Psychol Gen 2014 Aug;143(4):1457-1475. [CrossRef] [Medline]
  66. He J, Wang W, Huang M, Wang S, Guan X. Bayesian inference under small sample sizes using general noninformative priors. Mathematics 2021 Nov 05;9(21):2810. [CrossRef]
  67. van Doorn J, van den Bergh D, Böhm U, Dablander F, Derks K, Draws T, et al. The JASP guidelines for conducting and reporting a Bayesian analysis. Psychon Bull Rev 2021 Jun;28(3):813-826 [FREE Full text] [CrossRef] [Medline]
  68. Lee MD, Wagenmakers EJ. Bayesian Cognitive Modeling: A Practical Course. Cambridge, UK: Cambridge University Press; 2014.
  69. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [CrossRef] [Medline]
  70. Karyotaki E, Klein AM, Ciharova M, Bolinski F, Krijnen L, de Koning L, et al. Guided internet-based transdiagnostic individually tailored Cognitive Behavioral Therapy for symptoms of depression and/or anxiety in college students: a randomized controlled trial. Behav Res Ther 2022 Mar;150:104028 [FREE Full text] [CrossRef] [Medline]
  71. Merikangas KR, Calkins ME, Burstein M, He JP, Chiavacci R, Lateef T, et al. Comorbidity of physical and mental disorders in the neurodevelopmental genomics cohort study. Pediatrics 2015 Apr;135(4):e927-e938 [FREE Full text] [CrossRef] [Medline]
  72. Dopmeijer JM, Nuijen J, Busch MC, Tak N. Monitor Mentale gezondheid en Middelengebruik Studenten hoger onderwijs. Deelrapport I. Mentale gezondheid van studenten in het hoger onderwijs [Screening of mental health and substance use of students in higher education. Subreport 1. Mental health of students in higher education]. Rijksinstituut voor Volksgezondheid en Milieu RIVM. 2021.   URL: [accessed 2022-12-06]
  73. Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol 1999 Dec;52(12):1143-1156 [FREE Full text] [CrossRef]
  74. Rosbach M, Andersen JS. Patient-experienced burden of treatment in patients with multimorbidity - a systematic review of qualitative data. PLoS One 2017 Jun 23;12(6):e0179916 [FREE Full text] [CrossRef] [Medline]
  75. Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review. Contemp Clin Trials Commun 2018 Sep;11:156-164 [FREE Full text] [CrossRef] [Medline]
  76. Kasenda B, von Elm E, You J, Blümle A, Tomonaga Y, Saccilotto R, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA 2014 Mar 12;311(10):1045-1051 [FREE Full text] [CrossRef] [Medline]
  77. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008 Sep 29;337:a1655 [FREE Full text] [CrossRef] [Medline]
  78. Rahmadiana M, Karyotaki E, Schulte M, Ebert DD, Passchier J, Cuijpers P, et al. Transdiagnostic internet intervention for Indonesian university students with depression and anxiety: evaluation of feasibility and acceptability. JMIR Ment Health 2021 Mar 05;8(3):e20036 [FREE Full text] [CrossRef] [Medline]
  79. Bendig E, Bauereiß N, Buntrock C, Habibović M, Ebert DD, Baumeister H. Lessons learned from an attempted randomized-controlled feasibility trial on "WIDeCAD" - an Internet-based depression treatment for people living with coronary artery disease (CAD). Internet Interv 2021 Apr;24:100375 [FREE Full text] [CrossRef] [Medline]
  80. Krusche A, Rudolf von Rohr I, Muse K, Duggan D, Crane C, Williams JM. An evaluation of the effectiveness of recruitment methods: the staying well after depression randomized controlled trial. Clin Trials 2014 Apr;11(2):141-149 [FREE Full text] [CrossRef] [Medline]
  81. Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev 2018 Mar 22;2(2):MR000013 [FREE Full text] [CrossRef] [Medline]
  82. Healy P, Galvin S, Williamson PR, Treweek S, Whiting C, Maeso B, et al. Identifying trial recruitment uncertainties using a James Lind Alliance Priority Setting Partnership - the PRioRiTy (Prioritising Recruitment in Randomised Trials) study. Trials 2018 Mar 01;19(1):147 [FREE Full text] [CrossRef] [Medline]
  83. Lokman S, Leone SS, Sommers-Spijkerman M, van der Poel A, Smit F, Boon B. Complaint-directed mini-interventions for depressive complaints: a randomized controlled trial of unguided web-based self-help interventions. J Med Internet Res 2017 Jan 04;19(1):e4 [FREE Full text] [CrossRef] [Medline]
  84. Weisel KK, Zarski AC, Berger T, Krieger T, Moser CT, Schaub MP, et al. User experience and effects of an individually tailored transdiagnostic internet-based and mobile-supported intervention for anxiety disorders: mixed-methods study. J Med Internet Res 2020 Sep 16;22(9):e16450 [FREE Full text] [CrossRef] [Medline]
  85. Koelen JA, Vonk A, Klein A, de Koning L, Vonk P, de Vet S, et al. Man vs. machine: a meta-analysis on the added value of human support in text-based internet treatments ("e-therapy") for mental disorders. Clin Psychol Rev 2022 Aug;96:102179 [FREE Full text] [CrossRef] [Medline]
  86. Karyotaki E, Efthimiou O, Miguel C, Bermpohl FM, Furukawa TA, Cuijpers P, Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis. JAMA Psychiatry 2021 Apr 01;78(4):361-371 [FREE Full text] [CrossRef] [Medline]
  87. Wagenmakers EJ, Love J, Marsman M, Jamil T, Ly A, Verhagen J, et al. Bayesian inference for psychology. Part II: example applications with JASP. Psychon Bull Rev 2018 Mar;25(1):58-76 [FREE Full text] [CrossRef] [Medline]
  88. Makowski D, Ben-Shachar MS, Lüdecke D. bayestestR: describing effects and their uncertainty, existence and significance within the Bayesian framework. J Open Source Softw 2019 Aug 13;4(40):1541 [FREE Full text] [CrossRef]
  89. R Core Team, Bivand R, Carey VJ, DebRoy S, Eglen S, Guha R. foreign: Read Data Stored by 'Minitab', 'S', 'SAS', 'SPSS', 'Stata', 'Systat', 'Weka', 'dBase', ... v. 0.8-83. The Comprehensive R Archive Network. 2022.   URL: [accessed 2022-12-06]
  90. Wickham H. ggplot2: Elegant Graphics for Data Analysis. Cham, Switzerland: Springer; 2016.
  91. Harrell Jr FE. Hmisc: Harrell Miscellaneous. The Comprehensive R Archive Network. 2022 Nov 18.   URL: [accessed 2022-12-06]
  92. Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research. R package version 2.2.3. The Comprehensive R Archive Network. 2022 Sep 29.   URL: [accessed 2022-12-06]
  93. Bolinski F. ICare Prevent NL - lessons learned. Open Science Framework. 2022 Jun 10.   URL: [accessed 2022-10-06]

BF: Bayes factor
CAU: care as usual
CI: credibility interval
CONSORT-EHEALTH: Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth
GAD: generalized anxiety disorder
GAD-7: 7-item Generalized Anxiety Disorder Scale
GP: general practitioner
MDD: major depressive disorder
MINI: Mini International Neuropsychiatric Interview
PHQ-9: 9-item Patient Health Questionnaire
QIDS-CR: Quick Inventory of Depressive Symptomatology-Clinician Rated
RCT: randomized controlled trial
SIGH-A: Structured Interview Guide for the Hamilton Anxiety Rating Scale

Edited by A Mavragani; submitted 26.07.22; peer-reviewed by C Benjet, S Adam; comments to author 18.08.22; revised version received 02.09.22; accepted 13.09.22; published 14.12.22


©Felix Bolinski, Annet Kleiboer, Koen Neijenhuijs, Eirini Karyotaki, Reinout Wiers, Lisa de Koning, Corinna Jacobi, Anna-Carlotta Zarski, Kiona K Weisel, Pim Cuijpers, Heleen Riper. Originally published in the Journal of Medical Internet Research (, 14.12.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.