In recent years, there has been an increased emphasis on the role of schools in supporting children’s mental health [1-3]. This enhanced focus has been driven in large part by an apparent increase in mental health difficulties (including behavioral, social, and emotional difficulties) present in school-aged populations [4-6]—a concern that became increasingly prominent in the context of the COVID-19 pandemic and the associated school closures and social distancing measures [7,8]. There is also a growing recognition of the many unique advantages of using the school setting to promote and protect pupil mental health [9]. First, most lifetime disorders begin during the schooling years [10], which suggests that schools may be an ideal setting for early identification and intervention. Second, schools have access to most children, meaning that they are an important component of any public health approach to address child mental health difficulties [11-14]. Third, schools benefit from prolonged engagement with pupils, which can facilitate the implementation of mental health promotion and prevention strategies as well as support and interventions for pupils with identified mental health needs [12]. Finally, mental health support in schools is often more accessible to families than other types of support [15].

However, while school staff are increasingly expected to support children’s mental health [1], many do not feel prepared to do so [16-19] due in part to receiving limited training and supervision in this area [20]. Therefore, improving school staff’s confidence and preparedness are important considerations for supporting them in taking an expanded role in pupil mental health [21]. Most schools offer some form of mental health training [22,23], but many staff members believe that they could benefit from additional training [18-20,24-26]. One area where staff training may be particularly beneficial is the identification of and first response to pupils who have mental health difficulties or who are believed to be at increased risk of developing them. However, although there is evidence suggesting that school staff, parents, and practitioners see such training as an acceptable, feasible, and potentially useful way to support pupil mental health [20,27-29], empirical evidence for the effectiveness of such training is limited and focuses primarily on intermediate outcomes (eg, staff knowledge and confidence) rather than downstream outcomes (eg, accurate identification, access to support, and mental health outcomes) [30,31]. Furthermore, there are several potential barriers to implementing training programs in schools, including time, cost, and resource requirements [28].

Training programs that address these barriers may be beneficial for supporting schools in identifying and responding to pupil mental health difficulties. Brief, interactive web-based training programs are a particularly promising avenue as they have the potential to be more affordable, flexible, and scalable than other training formats. One such training is At-Risk for Elementary School Educators (hereinafter, At-Risk), a virtual simulation-based program developed by the American company Kognito [32]. The program, which has been completed by >125,000 teachers in the United States, aims to improve pupil mental health by “[building] awareness, knowledge, and skills about mental health, and [preparing] users to lead real-life conversations with pupils, parents, and caregivers about their concerns and available support” [33].

The program addresses many common implementation barriers to school-based mental health training. For example, At-Risk only requires approximately 1 hour to complete, which is much shorter than many other available training programs [31,34]. This comparatively low time commitment may address the concern that training programs are overly time intensive and, thus, make the training more feasible for busy schools [28,34,35]. The web-based format of At-Risk may also address concerns about school-based mental health programs being resource intensive [28]. Nearly all school mental health training programs documented in the literature are face-to-face sessions led by external facilitators [34,36], with only a few examples of web-based training [37-39]. For schools with limited budgets, programs requiring external facilitators can prove unsustainable and have limited scalability. In terms of financial resources, the costs of At-Risk vary depending on the number of licenses purchased, but the maximum cost is approximately £22 (US $30) per user, a price point that is feasible for many UK schools. In the United States, there have been many examples of bulk purchases at the district or state level that have made the training even more affordable per teacher. In many areas, the training is even free at point of use due to state- or district-wide licensing agreements [40].

To date, 3 randomized studies have examined the effectiveness and acceptability of At-Risk among samples of American teachers [17,41] and teachers in training [42] across school years. Each study found high satisfaction ratings, with between 75% and 85% of participants rating the training as useful, well constructed, relevant, and easy to use, and nearly all (88%-95%) reporting that they would recommend it to colleagues. The training also improved teachers’ self-rated preparedness, self-efficacy, and likelihood of identifying and discussing concerns about pupils’ mental health and referring them to appropriate support when needed. These improvements were reflected in the teachers’ behaviors—compared with teachers in the control group, those who completed At-Risk self-reported significantly more helping behaviors (eg, identifying psychological distress, discussing concerns with pupils and parents, and consulting with parents about options for care and support) and gatekeeping behaviors (ie, connecting pupils with care and support) after the training and at 3 months after the training. The findings of these studies indicate that At-Risk may help improve teachers’ ability to identify and respond to pupil mental health needs and lead to positive behavior change in terms of discussing concerns and facilitating access to care and support.

These 3 studies suggest that At-Risk may be a promising intervention for improving children’s mental health; however, there is still much to be learned about the training’s effectiveness, feasibility, and acceptability. Furthermore, to date, no evaluation of the training has been conducted outside the United States. There is increasing focus on the influence of context on the effectiveness of complex interventions [43-48], and while some interventions have shown success in terms of transportability [48], other interventions that have evidence of effectiveness in one context have demonstrated null or even negative effects in another [46]. Furthermore, information that could inform “transportability” is often not collected as part of evaluations [44], making it difficult to determine the likelihood of success in a new setting.

There are many contextual differences between the United States and the United Kingdom that could mean that school-based interventions developed in one country may not translate well to the other. Cross-country differences in education systems and (mental) health services are particularly relevant to this study. Differences in the education system include the length and content of initial teacher training, the number and roles of teaching assistants (TAs), and school funding structures. There are also key differences in the structure and availability of school-based mental health provision. In the United States, schools often have staff whose sole or at least main responsibility is mental health, such as school psychologists. While these roles are becoming more common in the United Kingdom with the implementation of the Green Paper recommendations [1], in most UK primary schools, mental health is included within the broader roles of the special educational needs coordinator (SENCo) and pastoral team. Finally, differences in the wider health care systems across the countries also mean that the process and outcomes of external referrals to specialist mental health services vary across settings, another fact that may influence the transportability of school-based interventions such as At-Risk.

Given these uncertainties regarding intervention transportability, additional evaluation of At-Risk is needed to understand whether it is a potentially useful and feasible tool to improve the identification of and response to mental health difficulties in UK primary schools. To explore the potential value of the training in this new context, we conducted a mixed methods feasibility study of At-Risk in 6 UK primary schools covering pupils aged 4 to 11 years. We aimed to examine the influence of At-Risk on staff confidence and preparedness, identification of pupils with mental health difficulties or increased risk of developing mental health difficulties, mental health support outcomes for identified children, and intervention acceptability and practicality.

We used a mixed methods, nonrandomized, pretest-posttest study design to explore the feasibility of At-Risk in UK primary schools. While feasibility studies are acknowledged as a key stage of intervention design and evaluation [49,50], there is no universally agreed-upon definition of a feasibility study [50,51]. Therefore, we focused on 3 criteria from the guidance by Bowen et al [52]: acceptability, practicality, and limited effectiveness testing.

At-Risk is a web-based training that is delivered individually and requires only a log-in and internet connection. Using a simulation-based teaching model, the training aims to (1) improve mental health awareness and knowledge, (2) empower users to approach pupils about what they have noticed, (3) impart skills to have meaningful conversations with pupils and parents, and (4) train users to refer pupils to further support. The diagram in Figure 1 illustrates how the training might lead to improved mental health outcomes for pupils.

The simulation begins with an introduction by a virtual coach, who defines and explains how to recognize the warning signs of psychological distress and specific mental health difficulties and provides guidance and practical advice for discussing and acting upon concerns. Users then practice 2 virtual scenarios. The first scenario involves a fifth-grade (UK Year 6; ages of 10-11 years) teacher speaking with the parent of a pupil showing signs of behavioral difficulties. The second involves a third-grade (UK Year 4; ages of 8-9 years) teacher speaking with a pupil showing signs of emotional difficulties. During the conversations, users choose what to say via drop-down menus organized into categories (eg, “bring up concerns” or “ask a question”) and phrases (eg, “Mia sometimes seems a little agitated in class”). Throughout the conversation, users receive feedback through a “comfort bar” (based on how the pupil or parent perceives the conversation), opportunities to “see” the thoughts of the pupil or parent, and suggestions from the virtual coach.

Importantly, there is no one “right” way to conduct the conversations, and several approaches can lead to a positive outcome. Throughout the conversation, users can “undo” actions to backtrack after receiving an undesirable response or to explore what the response would have been had they chosen another option. At the end of each conversation, the pupil or parent provides feedback on the conversation. The training finishes with a short segment on connecting pupils with further support.

For this feasibility study, we used an unmodified version of the training (ie, the standard training designed for American schools, not tailored to the UK context) provided free of cost by Kognito. The potential need for adaptation and tailoring was an important consideration that we explicitly examined as part of our exploration of the acceptability and practicality of At-Risk in this new setting.

This study was approved by the University of Cambridge Psychology Research Ethics Committee (PRE 2019.076). We obtained active informed consent from all teachers and TAs who took part in the study. We used an opt-out model for parental consent whereby parents received (directly from the schools via their preferred communication routes) an information sheet detailing study aims, procedures, how data would be used, and the right to opt their child out of participation. Parents had 2 weeks to opt their child out of the study by returning a hard copy of the opt-out form or emailing or calling the school. Schools kept track of all opt-outs and instructed teachers and TAs not to include these children in their forms. All quantitative data were collected using anonymous pupil and staff identifiers generated by the participating schools, and all qualitative data were deidentified before analysis, with identifiable information stored on secure servers at the University of Cambridge. Teachers and TAs received £20 (approximately US $28) vouchers for completing the training and questionnaires for at least 2 of the 3 time points and an additional £10 (approximately US $14) for taking part in an interview. School staff members who created the anonymous identifiers received £10 (approximately US $14) vouchers to thank them for their time.

This study offers the first UK evidence for Kognito’s At-Risk for Elementary School Educators, extending findings from 3 US-based trials and providing needed evidence regarding the potential utility, acceptability, and practicality of brief, interactive web-based mental health training for school staff. Overall, the findings showed that At-Risk is a feasible means of improving the identification of and response to pupil mental health difficulties in UK primary schools. Quantitative findings showed that staff preparedness and self-efficacy in identifying and responding to mental health difficulties increased after the training. Identification rates did not increase (and, in fact, decreased at the 3-month follow-up), but there was some suggestion that teachers’ and TAs’ identification became slightly more accurate in comparison with SDQ scores. Crucially, for those pupils identified as having mental health difficulties or increased risk, in-school mental health support outcomes (ie, documentation or discussion of concerns, conversations with pupils and parents, and in-class and in-school support) increased after the training, but more “downstream” outcomes (ie, documented SEMH status and referral and access to external mental health services) did not. Qualitative findings indicated that participants generally found the training acceptable and practical, with many explaining how they intended to use or had already used the skills they learned to improve their practice. Participants also suggested several useful improvements for the training and its implementation, including making it more relevant to the United Kingdom, adding more scenarios, and including a live element in the implementation of the training.

Findings regarding confidence and preparedness reflect those of the 3 US-based studies of At-Risk [17,41,42] and the wider literature surrounding teacher mental health training [31]. In general, mental health training seems to be effective in improving staff confidence. For example, 2 Australian-based studies [37,78] found that secondary school teachers who completed training felt more confident discussing their concerns and helping pupils with their mental health. Another UK-based study of a psychoeducational training program to improve recognition of depression in secondary schools [79] found significant pretest-posttest improvements in teacher confidence in their knowledge of symptoms, ability to recognize symptoms, and knowledge about how to speak with pupils about their mental health. However, not all studies have shown an impact, with a prominent UK-based study of mental health first aid training finding no effect on educators’ confidence in helping pupils with their mental health [80].

The general decrease in the proportion of pupils identified as having mental health difficulties or increased risk stands in contrast to previous studies of At-Risk, which found that school staff identified significantly more pupils of concern after completing the training [17,41]. Evidence of the effect of other training programs on identification is extremely limited [30,31,36], and differences in context, training content and delivery, baseline knowledge, and outcome measurement make it difficult to compare findings across studies. Two vignette-based studies showed little effect of either mental health first aid [78] or psychoeducational [81] training on identification (although each study also reported high recognition of difficulties before the training), whereas studies focused on real-world identification have shown mixed results [79,82]. However, changes in the proportion of identified pupils must be contextualized within the accuracy of identification. There are consequences of both over- and underidentification [83,84], most notably in terms of inefficient allocation of limited mental health support resources. While comparison with the SDQ suggested that there was some improvement in terms of the accuracy of identification following the training, underidentification remained a substantial challenge, with between one-half and two-thirds of pupils with elevated SDQ scores remaining unidentified by teachers and TAs. The underidentification of children with mental health difficulties in educational settings, particularly for children with internalizing as opposed to externalizing problems [85], has been well documented in the literature [30], and it is likely that a combination of identification models is required to address this challenge [27,29].

Promisingly, the training appeared to be useful in terms of connecting pupils with care and support, an outcome not frequently measured in other studies [30,31,34]. First, the findings suggested that participants had conversations about or documented concerns for a greater proportion of identified pupils following the training, which reflects findings from previous studies of At-Risk [17,41]. This is a rather unique outcome in the literature as other training evaluations have found no difference between training and control groups in terms of conversations with pupils and colleagues [78]. Importantly, this study went beyond conversations to include outcomes pertaining to in-school and external support. The increases in in-class and in-school support for identified pupils reflect findings of the UK-based study by Kidger et al [80] of mental health first aid training and the Australian-based pilot study by Parker et al [37] of a web-based training program, each of which found a positive effect of the training on helping behaviors. Although in-class and in-school support seemed to increase following the training, it is notable that referrals and access to specialist services did not. There are several plausible explanations for this finding. For example, it is likely that school staff were already aware of children with the most severe mental health difficulties and were confident and able to support newly identified pupils—who might have had lower-level mental health difficulties—within the school setting. However, if the training did lead to the identification of children who might benefit from specialist care, there are many barriers to accessing such support (eg, availability and long waiting lists) that might have influenced these outcomes, as reflected in both the qualitative interviews and the wider literature [23,86].

In addition, quantitative and qualitative findings suggested that the program was a good fit for individuals and schools, which aligns with previous research on the acceptability and perceived need for mental health training for school staff [18,20,27-29,87,88]. The training’s format seemed to be a key contributor to its feasibility. With a few exceptions [37,39,89], the web-based simulation-based format of At-Risk is unique among training programs and is well aligned with teachers’ preferences. For example, in their focus group study of UK secondary school teachers, Shelemy et al [20] found that participants wanted engaging, interactive, and concise training that included practical strategies and illustrative case studies, all of which are central to At-Risk. While the authors found that teachers disagreed over the usefulness of web-based training, it is possible that these concerns would have decreased during the COVID-19 pandemic as staff became more accepting of web-based opportunities to learn.

Qualitative findings also demonstrated the importance of school context and culture, which have been highlighted in previous research [27]. In particular, participants noted the importance of school culture in adopting mental health interventions into regular practice. In their systematic review, Moore et al [90] identified school culture, values, and policies as key facilitators of sustaining mental health interventions. A related area of focus was support from the SLT. This support is a well-recognized factor contributing to intervention success and sustainability for several reasons, including these leaders’ practical role in communicating about interventions and allocating specific time and resources to them [43,90,91]. However, it is important to recognize that mental health training for school staff may be even more needed and impactful in schools where mental health is not as much of a priority.

Our mixed methods approach, wide range of outcomes, and diverse sample of participating schools offer rich information regarding the feasibility of At-Risk in the United Kingdom. These strengths notwithstanding, there are also several limitations to consider when interpreting the findings. The nonrandomized design, while common for feasibility studies, prevents any conclusions regarding causality and also limits the exploration of other factors that may have influenced outcomes (eg, providing teachers and TAs with the Mental Health Resource Maps or SENCos and mental health leads with feedback on identified pupils). In terms of recruitment and retention, the study had 50% (54/108) attrition. Several factors may have influenced this, including the increased pressure on school staff due to the COVID-19 pandemic, the timing of the study within the school year, and the requirement to communicate with participants only via the study link person. While we tried to explore the effect of attrition through a complete case sensitivity analysis, we lacked important information on the characteristics of those who dropped out as this information was collected only at T2. Furthermore, we were only able to recruit 8 staff members for the posttraining interviews, which was far below our recruitment target. Low participation rates could again be due to several factors, including the impact of the COVID-19 pandemic or competing priorities. Of note, we were not able to recruit anyone who did not complete the training, any headteachers, or any staff from 2 of the schools (schools A and D). This could mean that we lack viewpoints that may be important for understanding the feasibility and utility of the training.

There were also limitations associated with the study measures. While the Teacher and TA Identification Form was informed by the literature and reviewed by our primary school staff advisory group, its validity and reliability are unknown. In addition, the questionnaire only measured mental health support outcomes for those pupils identified as having mental health difficulties or increased risk. Therefore, we do not have information on those who were not identified. The measure is also based on teacher and TA reports and so may not have complete information about all types of support that pupils receive. Another important limitation pertains to the comparator used to assess the identification outcomes. To understand the potential utility of the training program, it is important to have a robust comparator. While we chose to use the teacher-report SDQ, it would also have been interesting to compare identification outcomes with parent-rated mental health difficulties, particularly in light of the low interrater agreement of common measures of child mental health difficulties [92]. An even stronger comparator would be to assess the teacher and TA identification outcomes against a clinical interview; however, this was not feasible in this study.

Finally, at the time of writing, At-Risk is currently not available for use as Kognito restructures its offerings. This demonstrates a trend that unfortunately is a common occurrence in the field of mental health, whereby many evidence-based digital tools are not available to potential end users [93]. Nonetheless, the learnings from this feasibility study offer rich information on what type of content and format may be useful for training programs in this area and, as such, can support further development and evaluation in the field.

Studies have consistently demonstrated that school staff would appreciate additional training on how best to support pupil mental health [18,20,87,88]. However, to be scalable, such programs must be realistic in terms of time, cost, and resource requirements [28,90,91]. Contextualized within the wider literature on school-based mental health interventions, the findings from this study suggest that mental health training is a feasible option for upskilling school staff to identify and respond to pupil mental health difficulties. They further highlight several specific factors that might positively contribute to feasibility and scalability, many of which are reflected in the broader literature on mental health training [20,28]. For example, teachers and TAs appreciated that the training actively engaged them in learning and applying new skills and that it used realistic examples to demonstrate the real-world applicability of the training, whereas school leaders identified the relatively low time and cost requirements and flexibility as key factors that could make the training feasible for their school context.

However, this is not to say that there are no implementation barriers associated with At-Risk or similar training programs. While the resources required to implement At-Risk are relatively low compared with other training programs, they must still be considered within the context of other school priorities. As demonstrated in the interviews and the wider literature [3,43,90,91], support from school leadership is essential for securing the time and budget required to implement a training such as At-Risk, and in schools where mental health is not a priority, there are likely to be many barriers to implementation. Even in schools with strong support from the leadership team, it may be difficult to find the requisite budget, time, and human resources to devote to the training. Finally, as is the case with any school-based mental health intervention, it is important that schools do not take sole responsibility for pupils’ mental health. Active partnership between schools and mental health services is key to ensuring that schools feel empowered and supported in this role [21,90,94]. While the schools in this study worked hard to support pupils as best they could, interviewees expressed frustration about the difficulty of accessing external support for children who could benefit from it. This is not an uncommon theme in the wider literature surrounding school-based interventions [20,23,91] and is a key consideration for scaling up training programs.

The promising findings of this study suggest that additional research is needed to explore the role of scalable mental health training in supporting schools to protect and promote children’s mental health. On the basis of gaps in the literature, particular areas of interest include training for primary school staff (as most are focused on secondary school staff), web-based training (as opposed to traditional time- and resource-intensive in-person training), and training that takes a “whole school approach” by including all school staff members (rather than only teachers). This final area is especially interesting as findings from this study and others [27] have highlighted stakeholders’ preference that training programs include all school staff members. While our study jointly analyzed findings for teachers and TAs, future research would do well to consider how the unique roles and perspectives of these professionals—as well as other staff members within the school setting—might influence outcomes. Furthermore, future research should be more inclusive about their choice of outcomes, as too often evaluations of school staff training programs have focused on intermediate outcomes such as knowledge or confidence [31] without considering more “downstream” outcomes such as access to support. Finally, as demonstrated in our study, there is great value in using mixed methods approaches and including information about wider issues of feasibility and implementation, and studies that take this broader lens can help identify programs that are scalable, sustainable, and effective.

School staff would welcome additional mental health training to enable them to respond to pupil mental health difficulties, but there are many barriers to implementing such training at scale. Therefore, training programs that have relatively low time and resource requirements have great potential to fulfill an unmet need in schools. This mixed methods feasibility study showed that At-Risk for Elementary School Educators—an example of a brief, interactive web-based training program—is a feasible means of empowering school staff to accurately identify and respond to pupil mental health difficulties and increased risk.