A randomized trial using a counterbalanced design was employed to deliver YouthCHAT screening either before or after face-to-face HEEADSSS assessment.

Although YouthCHAT and HEEADSSS contained similar areas of assessment, it was necessary to map specific domains to each other for comparison of results. HEEADSSS includes domains on substance misuse, problem eating, sexual health, and physical activity, which are approximately equivalent to YouthCHAT modules. Questions regarding problems at home and safety were mapped for comparison (see Table 1). Although HEEADSSS mental health domain includes a number of nonspecific items such a sadness, grief, and difficulty sleeping, YouthCHAT mental health modules only comprised screening measures for depression and anxiety.

For HEEADSSS assessment, there are no threshold scores—the assessor decides whether the response is positive for that domain or not. For YouthCHAT, responses are clearly positive or negative for each domain. Where there are added tools, cutoff points are as follows:

All year 9 (13- to 14-year-old) students at a low-decile high school in Auckland, New Zealand, were invited to participate following the provision of written information about the study at the start of the school year and the completion of paired informed parental consent (using an opt-out process) and individual participant assent (as all students were aged <16 years). No students were excluded from the study. HEEADSSS assessment is mandatory for all year 9 students regardless of the study.

Participants were randomized to receive either HEEADSSS assessment by a school nurse followed by an electronic YouthCHAT screen (condition 1) or YouthCHAT followed by HEEADSSS assessment (condition 2). Clinic staff were provided with a random list from a computer-generated random numbers table, with consecutive sampling until all enrolled students had completed assessment. This took place during a planned break from class time when students receive their annually required HEEADSSS assessment. Review of results and any necessary follow-up was arranged by the school nurse immediately following the completion of YouthCHAT screening and HEEADSSS assessment.

Primary outcome measures were (1) the time taken to complete YouthCHAT and HEEADSSS, (2) comparative detection rates for YouthCHAT and HEEADSSS for each issue, and (3) acceptability of YouthCHAT to students and staff. YouthCHAT data were collected electronically (completed on an electronic tablet by students), and encrypted results were securely stored on a central database. HEEADSSS results were entered into the electronic health record by school nurses. A subset of students completed paper-based acceptability questionnaires, and the 3 school nurses were interviewed individually.

Quantitative data were analyzed using Microsoft Excel 2013 and Statistical Package for the Social Sciences version 25 (SPSS v25). Analyses included basic descriptive statistics, between-intervention analyses undertaken with paired t tests (for numeric variables) or McNemar tests (for categorical variables), and between-condition nonparametric analyses undertaken with Mann-Whitney U tests. Distributions were checked for normality throughout.

Qualitative data were analyzed using a general inductive approach [31], with collated text independently coded by 2 researchers (HT and FG) to identify emerging themes. Discrepancies were resolved through an adjudication session.

Further methodological details are described in our trial protocol [32]. The study was approved by the New Zealand Northern Region Ethics Committee (16/CEN/137/AM03).

From the 139 eligible students, 129 assented. Electronic screening and face-to-face assessments were conducted between March and November 2017. There were incomplete or missing data for 19 students, giving a total sample size of 110 for analysis (81%; see Figure 1).

In addition, 63% (71/113) of the participants were of Pacific ethnicity, 29% (33/113) were of Māori ethnicity, and the remaining 8% (9/113) were of New Zealand European or other ethnicity. Moreover, 51% (58/113) of the participants were male, and the randomized condition numbers were similar, with 49% in condition 1 and 51% in condition 2. From the 32 students invited to participate in a focus group (8 during each term), 21 (66%) attended, with 3 groups of 5 participants and 1 group of 6 participants.

HEEADSSS time data were missing for 19 students; therefore, the comparative time to complete analyses were conducted for 94 students. YouthCHAT took an average of 8 min 57 seconds (range 1 min 45 seconds to 54 min 15 seconds) to complete, compared with HEEADSSS with an average nearly double at 17 min 22 seconds (range 3 min to 45 min), giving a mean difference of 8 min 25 seconds (range 6 min 20 seconds to 11 min 10 seconds; P<.01). For several students, the Wi-Fi connection was lost for YouthCHAT, which may be reflected in the outlier durations of 25 min to 54 min, whereas the vast majority took 10 min or less.

The comparative detection rates are presented in Table 2. The 2 assessments had roughly similar detection rates for substance misuse and problems at home, but YouthCHAT detected significantly more issues around problems with eating or body image perception, safety, physical inactivity, and sexual health (all P<.01). HEEADSSS mental health category had a greater detection rate when compared with YouthCHAT-detected positive responses to the depression and anxiety tests (P<.01); however, no direct comparison between the assessments for only depression and anxiety rates was possible.

The order in which students received YouthCHAT and HEEADSSS (condition 1 vs condition 2) made no significant difference to the duration of YouthCHAT assessment. For YouthCHAT, condition 1 took an average of 8 min 37 seconds (SD 6 min 54 seconds) and condition 2 took an average of 9 min 24 seconds (SD 8 min 26 seconds). For HEEADSSS, condition 1 took an average of 16 min 13 seconds (SD 8 min 42 seconds) and condition 2 took an average of 18 min 7 seconds (SD 9 min 49 seconds).

Similarly, the order made no significant difference to YouthCHAT-positive depression PHQ-A screen rates (both condition 1 and condition 2 had 5 positive screens; χ²_0.95=0.0) or to anxiety GAD-7 rates (condition 1 had 3 screens and condition 2 had 2 screens; χ²_0.68=0.25).

The results of the student acceptability survey completed by 21 students are shown in Table 3. There were no significant differences.

A total of 4 key themes emerged from the analysis of the 3 nurse interviews (Table 4): (1) valuable tool, (2) difficulties with use, (3) comparing YouthCHAT with HEEADSSS, and (4) additional uses for YouthCHAT. In summary, students found YouthCHAT easy to understand, nurses liked its look and feel, it helped identify students at risk, and nurses found the summary report and the help question useful (Figures 2 and 3). Students did identify some difficulties with use, including Wi-Fi connectivity problems at times, and some students had literacy, language, or cognitive ability issues that were a barrier to its use. In comparison with HEEADSSS, nurses found YouthCHAT easier for students to answer, faster to administer, and helped students subsequently answer face-to-face questions. Nurses suggested additional uses, including repeating annually for a longitudinal picture and using opportunistically with at-risk students in all year groups.

Our results indicate that YouthCHAT is a timesaving, effective, and acceptable psychosocial screener for use in a high school setting. Even with the occasional Wi-Fi glitch, it was significantly faster to use.

In general, YouthCHAT had similar or significantly higher detection rates than HEEADSSS. Although the detection rate for mental health problems or distress was higher with HEEADSSS, this reflects poor mapping of the 2 assessments for this issue. YouthCHAT mental health consisted solely of positive scores for depression or anxiety on the PHQ-A or GAD-7, whereas the HEEADSSS domain also included many nonspecific issues such as low mood, distress, unresolved grief, sadness about a historical event, and difficulty sleeping; hence, it is to be expected that more students will score higher for mental distress than score positive for depression or anxiety with YouthCHAT. Rates of depression, anxiety, and substance use problems identified via YouthCHAT were in line with expectations for this age group [33,34].

Students revealed significantly more concerns via YouthCHAT than in their HEEADSSS assessment about sensitive issues such as their body weight and sexual health and safety issues such as bullying, violence, and anger. This is consistent with evidence that youth prefer to disclose sensitive information via electronic means, without fear of being judged [24-27]. Electronic screening helps them structure their thoughts and prioritize the issues for which they want help [24]. Despite the difference between groups not being statistically significant, students’ concerns regarding privacy of information and being asked questions about sensitive issues suggest that screening for psychosocial issues should always be undertaken in a careful manner and an appropriate setting.

To date, no other screening instrument has been shown to be effective for comprehensively identifying multiple psychosocial problems in young people. Reviews of individual instruments for identifying common psychological problems in young people have identified strengths and weaknesses of different psychometric tests and recommended that these instruments are reserved for targeted use within clinical settings [35,36]. Overall, 3 HEEADSSS-based electronic screeners, TickIT [37], myAssessment [38], and the Headspace Assessment Interview [39], have recently been demonstrated to be acceptable to users in hospital and youth clinic-based settings but have not been evaluated regarding their detection rates.

Given the temporally evolving and fluctuating nature of psychosocial issues during adolescence [40,41], routine (eg, annual) YouthCHAT screening is likely to increase the chance of early detection and intervention. Conducting YouthCHAT before a scheduled HEEADSSS assessment means that the latter needs to focus only on domains where YouthCHAT is positive. Such a targeted HEEADSSS approach will reduce the time taken and hence the cost of this assessment. Embedding YouthCHAT screening within a regular holistic school health check may also increase mental health literacy [42], normalize the management of psychosocial issues, and reduce stigma about help seeking [43]. Downstream benefits of early intervention may include improved social relationships, better engagement in education and employment, reduced involvement with the justice system, and lower rates of youth suicide [44].

Strengths of this study include the comparison of YouthCHAT with an existing means of evaluating young people for psychosocial problems, the high response rate, and the collection of both student and staff perspectives on the use of electronic screening within a school environment. The restriction of participants to 13- to 14-year-olds and 3 school nurses from a single high school limits the generalizability of our findings. Owing to the variability in the time taken to complete both tests being considerably less than anticipated (ie, smaller SDs for the time taken to complete YouthCHAT screening and HEEADSSS assessment than expected), our power to detect a difference between the 2 interventions was higher than anticipated. Furthermore, there is a clear statistical difference between the interventions based on this sample; therefore, our final sample size was sufficient to answer our primary research questions. The inclusion of predominantly Māori and Pacific Island participants is both a strength and weakness of this study. Māori and Pacific people comprise 20% and 11%, respectively, of New Zealanders aged 10 to 17 years; hence, these ethnicities are oversampled. However, Māori and Pacific Island youth have higher rates of emotional difficulties [45,46], including depression [47] and suicide [48], yet they access specialist services at lower rates than other ethnicities [49], so early identification and intervention for these youth is key. Finally, the inability to directly map all the YouthCHAT modules to the HEEADSSS assessment domains limited the scope of comparison.

YouthCHAT has been shown to be significantly quicker than HEEADSSS to administer, has a high detection rate of a range of psychosocial issues, and is acceptable to both students and staff. Its potential use is for both opportunistic and routine annual screening of high school students, especially those of low socioeconomic status. Next steps include its evaluation with students of different ages and in different types of school settings. Current evidence supports its use as a first-line screening instrument, which can be followed by a targeted HEEADSSS assessment where indicated.