Mental health conditions are a leading cause of health-related burden across the world, with anxiety and depression being ranked among the top 25 causes of burden in 2019 [1]. Both disorders are very common, affecting >29% of the global population in their lifetime [2]. The recent COVID-19 pandemic has accelerated this even further, raising the global prevalence by 27.6% between 2020 and 2021 for major depressive disorder and 25.6% for anxiety disorders [3], highlighting the need for effective and accessible mental health support.

Cognitive behavioral therapy (CBT) has been shown to be a highly effective treatment for depression and anxiety disorders [4-6] and is the first line of treatment for these conditions in many therapy guidelines worldwide [7,8]. Nonetheless, CBT does not benefit everyone. In fact, a recent meta-analysis of 409 trials reported that 58% of patients do not respond to treatment [5]. Several reasons for unsuccessful therapy have been identified. Among them, the most prominent is a failure to engage with therapeutic exercises and materials (ie, homework assignments) between therapy sessions [9].

Between-session support, such as completion of therapeutic materials and exercises, is a necessary component of CBT [10], allowing patients to maintain the progress made in the therapy and transfer it to real life to achieve short-term and long-term improvements [11]. Broadly, between-session work can be divided into psychoeducational (eg, reading materials about symptomatology), self-assessment (eg, monitoring feelings, thoughts, or behaviors), and modality-specific tasks (eg, exposure to images for specific phobias). Meta-analyses have consistently demonstrated that both the quantity and the quality of such task completion significantly predict treatment outcomes [12,13].

Thus, enhancing high-quality engagement with CBT materials and exercises between therapy sessions is a key objective for improving CBT treatment outcomes [14]. This is particularly critical in group therapy settings, where dropout rates are higher and recovery rates are lower than in individual therapy settings [15-17]. Traditionally, therapeutic materials and exercises in CBT have been distributed as worksheets in pen and paper format, downloadable PDF format, or delivered as digital applications, requiring the patient to engage with these static materials without any intelligent supervision. However, this method has shown limitations, often resulting in delayed completion or incomplete materials, highlighting the need for innovative solutions to enhance patient compliance and treatment effectiveness. One promising avenue for enhancing between-session support has been the deployment of digital tools [18]. Most notably, recent developments in generative artificial intelligence (AI) and large language models (LLMs), such as ChatGPT, have showcased that AI can enable a truly tailored user experience with interactions nearly indistinguishable from humans [19] and lead to increased engagement and treatment adherence [20,21]. Thus, the use of this novel technology holds great potential to support clinical practice, especially for supporting patients between therapy sessions to meaningfully engage with their course of treatment.

We developed a generative AI–enabled therapy support tool, Limbic Care, to support patients between sessions during clinician-led group-based CBT. The AI-enabled therapy support tool includes chat-based therapeutic materials to allow for more natural engagement with these materials than traditional, static CBT materials. Recognizing the important role of personalization in overcoming the barrier to the uptake of digital solutions [22], our solution uses generative AI to deliver a user-centric experience. Personalization addresses the diverse needs and preferences of individuals undergoing CBT rather than using the one-size-fits-all approach, which often falls short in addressing unique needs. It may also help cultivate a relationship between users and the AI-enabled therapy support tool. This relationship, also known as therapeutic alliance, has been shown to increase engagement and predict positive clinical change [23]. Thus, we hypothesized that the use of the AI-enabled therapy support tool increases engagement with therapy and thus leads to better outcomes for patients. This support was tailored and personalized, however, treatment was still provided by the clinician in line with medical device regulations [24]. In this observational study, we evaluated a novel solution for patient engagement and clinical outcomes. Assessing 244 patients undergoing CBT treatment in 5 of the United Kingdom’s National Health Service (NHS) Talking Therapies services, we found that patients who chose to use the AI-enabled therapy support tool during their treatment achieved better treatment success. Moreover, we showed that engagement with the AI-enabled therapy support tool was linked to increased reliable improvement, recovery, and reliable recovery, as well as more attended treatment sessions and reduced dropout from treatment. Therefore, our results suggest that the AI-enabled therapy support tool may improve individual patients’ outcomes and support clinical services by increasing treatment adherence and treatment effectiveness.

We evaluated the efficacy of an AI-enabled therapy support tool (Limbic Care; Limbic Ltd [25]) in a real-world clinical context in 5 NHS Talking Therapies for anxiety and depression services provided by Everyturn Mental Health in the United Kingdom. Specifically, the tool was implemented in the context of completing therapeutic materials and exercises between the sessions of a synchronous web-based group-based CBT program according to the NHS Talking Therapies for anxiety and depression manual [26].

The AI-enabled therapy support tool is a mobile app using AI to offer support between therapy sessions to patients in psychological talking therapy who are aged ≥18 years, under the supervision of a trained clinician.

The app features a conversational chatbot (Figure 1) to assist with the completion of therapist-assigned therapeutic materials and exercises, which are a component of CBT. The chatbot was powered by an LLM (GPT-4 [OpenAI] at the time of the study) to deliver clinically validated materials, that is, therapeutic materials and exercises developed by clinical experts and assigned by the treating clinician. Specifically, the clinical materials in the AI-enabled therapy support tool were interventions (interactive exercises helping the patient work through a presenting problem, eg, reframing negative thoughts) and psychoeducational materials (educational materials, eg, about CBT). Importantly, there were multiple safety layers and guardrails implemented to ensure that the LLM strictly adhered to the task at hand [27].

The therapeutic materials primarily consist of standard materials for CBT as they are also commonly used in NHS Talking Therapies. These are delivered in a conversational way to the patient using several custom-tailored LLM modules and supervisory AI tools.

All conversations were constantly monitored using several machine learning safety modules to ensure appropriateness, prevent harmful responses, monitor risks, and ensure regulatory compliance [27]. The conversations as well as these machine learning models were monitored and continuously improved by the company’s research team.

In contrast to traditional homework worksheets, the patient can interact conversationally to navigate through the materials and receive notifications regarding assignments. The app encourages and empowers patients to engage in the therapeutic process outside group therapy sessions and continue the progress made in therapy in their real lives as well as provides empathetic support for the user.

This was a multisite, real-world, observational study to investigate the impact of the AI-enabled therapy support tool on clinical outcomes. The AI-enabled therapy support tool was offered to all patients eligible for 11 group-based CBT groups starting between October 2023 and January 2024. All (N=244, 100%) patients had access to the same group sessions irrespective of their decision to use the AI-enabled therapy support tool between the sessions. The clinicians facilitating the group sessions were trained to offer the use of the AI-enabled therapy support tool in the first session of CBT. All (N=244, 100%) patients were encouraged to complete therapeutic materials and exercises between sessions regardless of which delivery method they chose.

We compared the clinical outcomes of individuals who signed up to use the AI-enabled therapy support tool (the intervention group) with those of individuals who did not (the control group). It is important to note that individuals who chose not to use the AI-enabled therapy support tool had access to CBT worksheets, which represent the standard way of completing homework between therapy sessions and which were also assigned by the treating clinician. Therefore, aside from the introduction of the AI-enabled therapy support tool, no other changes were made to the content or the delivery method of the sessions, ensuring that any observed effects can be attributed to patients choosing to use the tool. In addition, we investigated whether engagement with the AI-enabled therapy support tool, specifically the completion of therapeutic materials and exercises on the app, was linked to the outcome measures.

Clinical audit studies within the NHS Talking Therapies do not require additional patient consent or ethics approval, as determined by the NHS and in line with National Institute for Health and Care Excellence guidelines [28]. This exemption from additional ethical review was further confirmed by the UK Health Research Authority. In this case, a clinical audit was used to determine whether the introduction of a new technology would improve the standard of care. Patients voluntarily opted in to use the AI-enabled therapy support tool. Those who were offered access to the AI-enabled therapy support tool agreed to the privacy and data policy agreement in the app and consented to the use of their anonymized data for audits and research studies. Because this was a real-world observational study in group therapy settings, patients were not provided any financial compensation. Regarding the identification of individuals, treatment and engagement outcome data in this study were anonymous; therefore, it was not possible to identify individual patients.

The qualitative user experience study in a nonclinical sample was separate from the clinical audit and was approved by the University College London Research Ethics Committee (6218/003). Participants in this study were paid £4.20 (US $5.17), and this was not contingent on their use of the AI-enabled therapy support tool. Participation was voluntary, and all users provided informed consent. Free-text answers were anonymized for coding and analysis.

In addition to the impact of the AI-enabled therapy support tool on clinical outcomes, we were further interested in the perceived benefits of the tool and potential areas for improvement. Because it can be challenging to contact patients undergoing treatment and minimize the burden on them, we conducted a separate user study to assess user experience. Therefore, we recruited 113 individuals experiencing depression and anxiety symptoms (PHQ-9 score of ≥5 or GAD-7 score of ≥5) from the web-based platform Prolific [32] and instructed them to install the Limbic Care app. Participation was voluntary, and all (113/113, 100%) participants provided informed consent. Participants were not instructed on how to engage with the AI-enabled therapy support tool but rather were told they could use it as much or as little as they would like for the 4-week period of the study. After 4 weeks, participants were invited to answer some questions about their experience using the app, and only users who had used the app were included in the analysis. To probe the perceived benefits of the AI-enabled therapy support tool, participants were asked if they felt their mental health had benefited from using the app (“What benefits to your mental health (if any) have you gained from using the app?”). To understand potential areas for improvement, participants were asked if they had any feedback on how the app could be improved (“Do you have any feedback or suggestions on how we could improve the Limbic app?”). Participants were reimbursed £4.20 (US $5.24) for completing the study, and this was independent of their use of the AI-enabled therapy support tool.

Data were collected asynchronously, using free-text answers from a nonclinical, convenience sample on Prolific (n=113). Participants did not have a mental health diagnosis, though they experienced elevated symptoms of anxiety and depression (as indicated by scores of ≥5 for the PHQ-9 and GAD-7). Data were analyzed using reflexive thematic analysis, taking an inductive approach to coding [33], and a realist epistemological position, assuming that meaning and experience are reflected in language [34]. Two researchers (MR and LMD) independently coded a subset of responses and iteratively cross-checked interpretations and codes. Codes were refined using a comparative method, with new interpretations evaluated against existing ones. Inductive analysis of similar codes produced content themes highlighting potential benefits and areas for improvement in the AI-enabled therapy support tool.

Although the semiautomated nature of data collection precluded purposive sampling for saturation, no new codes emerged in later responses that altered the meaning of earlier findings, indicating data saturation was reached [35]. The themes and supporting participant quotes are presented in the Results section.

The coding team consisted of MR, a male researcher, and LMD, a female researcher. Both researchers, while predominantly quantitative, were trained in thematic analysis and had previous experience in qualitative research [36].

Credibility was ensured through reflexivity and investigator triangulation [37,38]. Data were collected anonymously and asynchronously, minimizing the influence of researcher characteristics on participant responses. Open-ended prompts encouraged both positive and negative reflections to capture a balanced perspective. While the nature of data collection did not allow methodological triangulation, investigator triangulation was achieved through the involvement of multiple researchers (MR, LMD, JH, and JM). To ensure transferability, we described the study’s context and the participants (thick description) to allow the readers to evaluate whether the findings are transferable to other care contexts [37,38]. Dependability and confirmability were ensured by thoroughly documenting the research steps from the start of the study to the interpretation and reporting of the findings [37,38].

We compared pretreatment data for patients in both groups to ensure that they did not differ based on demographic characteristics (age, gender, ethnicity, and sexual orientation), wait time to treatment, and depression and anxiety baseline symptom scores. In addition, recognizing that different referral methods can influence clinical outcomes, as illustrated by the AI-enabled self-referral tool increasing treatment success [21,39], we also compared the referral methods between groups. A 2-tailed chi-square test was applied for categorical outcomes, and a 2-tailed independent student t test was used for continuous outcomes.

Next, clinical outcomes were compared between the intervention and the control group. Linear regression models were run for continuous outcomes (number of attended appointments and proportion of DNA sessions), and logistic regressions were run for categorical outcomes (dropouts, reliable improvement, recovery, and reliable recovery). In both cases, the outcomes of interest were entered as independent variables and the intervention group (intervention group=1; control group=0) as a dependent variable. In addition, we controlled for baseline anxiety and depression symptom scores and the proportion of women and heterosexual participants. For linear regression, we reported regression coefficients, along with the 95% CIs, and for logistic regressions, we reported odds ratios (ORs) along with 95% CIs.

Use outcomes were investigated within the group with access to the AI-enabled therapy support tool. Specifically, we were interested in investigating if there would be any association between patients’ level of engagement with the app and their clinical outcomes, in line with previous work suggesting that the completion of therapeutic materials and exercises and engagement between sessions may lead to an improvement in treatment outcomes [14]. Pearson correlation was used for continuous variables, and point-biserial correlation was used for binary variables.

Patients (N=244) attending regular group therapy in Talking Therapies in the United Kingdom were offered to use our AI-enabled therapy support tool, of which 150 (61.5%) downloaded and used the AI-enabled therapy support tool (intervention group). The remaining 94 (38.5%) patients used static therapeutic materials, which were either in pen and paper format or downloadable PDF format (control group). We followed both groups from enrolling in the study until the end of their therapy (typically 8-10 weeks long). To ensure that the patients used the AI-enabled therapy support tool as intended, we calculated the percentage of individuals who engaged with the therapeutic materials at a certain number of weeks from the start of their therapy (known as app retention). We found that 79.3% (119/150) of the individuals were engaged at week 1, 64% (96/150) were engaged at week 2, 51.3% (77/150) were engaged at week 3, 42% (63/150) were engaged at week 4, 35.3% (53/150) were engaged at week 5, and 19.3% (29/150) were engaged at week 6.

To check for baseline differences, we compared pretreatment clinical and demographic characteristics. The mean ages of the participants in the intervention group (mean 40.3, SD 13.2 years) and the control group (mean 40.5, SD 15.5 years) were not significantly different (t₁₈₃=0.11; P=.91) nor was the ethnicity of the participants (intervention: 132/150, 88.7% White participants; control: 78/94, 83% White participants; χ²₁=0.4; P=.53). However, the groups differed significantly in the proportion of heterosexual individuals (intervention: 129/150, 86% heterosexual individuals; control: 69/94, 73% heterosexual individuals; χ²₁=5.4; P=.02) and women in the group (intervention: 111/150, 74% women; control: 58/94, 62% women; χ²₁=4.2; P=.04). Therefore, we controlled for the proportion of heterosexual individuals and women in all subsequent analyses. The 2 groups did not differ in anxiety (b=–0.11; P=.81) symptom scores nor in depression scores at baseline (b=−0.56; P=.36; Table 1). Nonetheless, we decided to control for baseline depression (PHQ-9) and anxiety (GAD-7) scores in all further analyses, as they are known as prognostic factors for changes in symptom severity [40,41]. Finally, the groups did not differ in the time they waited to start treatment (t₁₂₂=–0.32; P=.75) nor the proportion of individuals who referred using the AI-enabled self-referral tool (χ²₁=1.2; P=.28).

The themes and supporting participant quotes are presented in Table 3.

Of the 113 users, only 38 (33.6%) had suggestions for areas of potential improvement. The rest of the users had either no suggestions for improvement (n=63, 55.8%) or mentioned positive feedback even when queried about areas of improvement (n=12, 10.6%). Of the users who had suggestions for improvements, the most common comment (17/113, 15%) focused on the fact that while the chatbot was already very personalized, it was still possible to detect that it was not a human. In addition, 11 (9.7%) users had suggestions for specific additional features that they would either like to see or did not enjoy in the app. There were 4 (3.5%) users asking for more tailored notifications, and 3 (2.7%) users found that the additional guard rails to ensure patient safety restricted their user experience.

This indicated that, overall, users suggested few areas for improvement, with many of these dimensions being highly idiosyncratic to individual users.

We investigated the effects of a generative AI–enabled therapy support tool on treatment success and treatment adherence for patients undergoing group therapy in 5 of the UK NHS Talking Therapies provided by Everyturn Mental Health. A key part of therapy is the engagement with therapeutic materials that are assigned to the patient by the therapist. This engagement is crucial for applying tools and techniques learned in the therapy to real-life contexts [10]. In our study, the intervention group received in-between session support through an AI-enabled therapy support tool, while the control group completed the assignments using standard, static CBT worksheets.

Patients who had access to the AI-enabled therapy support tool showed significant improvement in clinical outcomes. Specifically, they achieved higher rates of reliable improvement, recovery, and reliable recovery compared to the control group. Similarly, patients using the AI-enabled therapy support tool attended more therapy sessions, were less likely to not show up to scheduled therapy sessions (that is, fewer DNAs), and were less likely to drop out from therapy. Importantly, the tool’s use level was significantly associated with treatment success. Particularly, higher engagement with the AI-enabled therapy support tool was associated with increased reliable improvement and better recovery and reliable recovery. Furthermore, the use of the AI-enabled therapy support tool was significantly associated with treatment adherence, that is, app use was correlated with a higher number of attended therapy sessions and with a lower proportion of missed appointments and treatment dropouts. Therefore, our findings suggest that integrating the AI-enabled therapy support tool into standard therapy may lead to positive outcomes at the individual level by improving clinical outcomes. Moreover, the tool may save precious clinical time by reducing missed appointments and through this may reduce costs for the services.

Our findings indicate that using the AI-enabled therapy support tool to support patients between sessions can improve treatment outcomes. In line with previous research indicating that more meaningful engagement leads to increased intervention effectiveness [42] and hence better treatment success [43], we found that the use of the AI-enabled therapy support tool was associated with increased reliable improvement, recovery, and reliable recovery–the 3 core metrics for treatment success that are routinely assessed in NHS Talking Therapies [26]. One key aspect of our AI-enabled therapy support tool is personalization, a crucial component of CBT [44], which means that patients receive tailored information and interactions to support them in understanding how the CBT exercises relate to their situations and daily problems. Cultivating a therapeutic relationship or therapeutic alliance has been shown to increase engagement and predict positive clinical change [23]. In the case of our AI-enabled therapy support tool, we encouraged this using an interactive chat with the patients, which provided personalized and individualized assistance. It is noteworthy that these effects seem to be specifically linked to the delivery of CBT exercises through generative AI rather than only by displaying CBT psychoeducation materials in a digitized format per se. Moreover, the qualitative analysis of user feedback suggests that these generative conversations were especially useful for helping users explore new perspectives and views on their problems, resulting in increased clarity. These findings suggest a unique role of generative AI in the delivery of CBT exercises.

It is also worth noting that higher engagement with the app was linked to more attended sessions and a lower proportion of reported DNA sessions. This indicates that increased engagement between sessions is not only beneficial to the individual patients but also to the services that experience approximately 12% fewer missed appointments and 42% fewer treatment dropouts [45]. Furthermore, considering the substantial cost per reliable recovery in low-intensity treatment in NHS Talking Therapies, estimated at £1087 (US $1355.91) [46], the increase of 21 percentage points observed in the AI-enabled therapy support tool group translates to substantial cost savings. Assuming that the AI-enabled therapy tool is solely driving this 21% improvement, it would translate to £228 (US $284.40) of generated value per patient based on improved recovery alone, making it highly cost-effective. While future studies should assess the cost-effectiveness of such tools in detail, our data indicate that enhancing treatment effectiveness within NHS Talking Therapies through AI-enabled therapy support could lead to substantial cost savings and improved quality of care.

There has been a large societal debate about the use of AI, and especially LLMs, in clinical settings [47], emphasizing the importance of caution when introducing novel technologies into health care settings. Importantly, the AI-enabled therapy support tool has undergone rigorous testing as part of an International Organization for Standardization 13485–compliant quality management system, ensuring the product has gone through gold-standard risk management processes. Moreover, the AI-enabled therapy support tool was only complementing standard care provided by trained clinicians. Thus, the AI-enabled therapy support tool is only populated with therapeutic materials chosen by the clinician rather than making any clinical decisions itself. While this is a low-risk task, the positive impact on treatment outcomes was nevertheless substantial. These findings exemplify the importance of critically evaluating how AI can be used safely and effectively in clinical settings.

Importantly, our study uses data acquired from real patients receiving treatment from the UK NHS Talking Therapies, which means that the presented data provide high ecological validity for real-world clinical settings. Another advantage of our study is that the AI-enabled therapy support tool offered personalization through generative AI, and a lack of personalization has been identified as a key barrier to the uptake of digital solutions in previous research [22]. Lastly, the novel AI-enabled therapy support tool targeted support between therapy sessions, which is one of the most prominent reasons for disengagement [9].

First, it is important to consider the observational design of this study. Following this promising observational data, a randomized controlled trial should be conducted. Second, while we were statistically controlling for several potential confounding factors (eg, patients’ symptom severity), we did not have access to and were therefore unable to control for certain aspects, such as socioeconomic status. Nonetheless, given that the participants in both groups were from the same services, and thus similar geographic areas, we are not expecting major differences in their socioeconomic background. Studies with randomized designs may help to control for such confounds and would be a valuable future research direction. Finally, while we incorporated qualitative feedback about the perceived benefits of the AI-enabled therapy support tool, as well as potential areas where the tool could be improved, these data were collected from a nonclinical convenience sample of users. While the users showed elevated levels of depression and anxiety symptoms, it is possible that their views do not align with those of the patient sample. To this end, we would aim to incorporate a mixed methods process evaluation as part of a future randomized controlled trial evaluating the effectiveness of the tool.

In conclusion, this study showed that integrating an AI-enabled therapy support tool can lead to positive outcomes for patients and clinical services alike, by improving therapeutic outcomes and reducing costs associated with missed or dropout sessions. These are important findings considering that AI-enabled digital mental health tools are an emerging field and that they have the potential to increase access to treatment for patients and lessen the burden on mental health services.