This report follows the protocol of Consolidated Standards of Reporting Trials involving AI (CONSORT-AI) [17]. All participants signed an informed consent form in person prior to participation.

The RCT protocol was reviewed and approved by the Ozark Center’s Research Committee, the Freeman Health System Institutional Research Board, and the State of Missouri Department of Mental Health under the research title “Optimizing behavioral healthcare delivery through technology.”

Participants in this study were 47 adults (34, 72% women and 13, 28% men; mean age 30.64, SD 11.02 years) referred to individual, outpatient CBT with a diagnosis of a depressive or anxiety disorder and the clinicians treating them. Exclusion criteria included having a severe physical or mental health condition that might interfere with treatment attendance or compliance. A total of 23 participants were randomized to the AI platform group and 24 to TAU. Most participants were White (44/47, 94%), and 6% (3/47) were Hispanic or Latino. Table 1 details participants’ demographic characteristics.

This study was conducted at the adult outpatient program at the Ozark Center, Missouri, United States. Potential participants were initially identified in person by the center's intake team and were randomized at the therapist level to either the Eleos or TAU group after consent. Therapists in both AI and TAU arms were employees of the clinic, all master’s degree–level social workers or counselors. The primary outcomes included the feasibility and acceptability of the AI platform. Secondary outcomes included changes in depression (Patient Health Questionnaire-9; PHQ-9) [18] and anxiety (Generalized Anxiety Disorder-7; GAD-7) [19]—scores as well as treatment attendance, satisfaction, and perceived helpfulness. Participants completed depression and anxiety assessments digitally at baseline, Month 1, and Month 2. Treatment satisfaction and helpfulness were assessed at the end of the trial using 2 items rated on a 1-5 Likert scale. Qualitative feedback from therapists in the AI group was collected at the end of the trial. All assessments were completed digitally. Data on session attendance were provided by the clinic. Figure 1 presents a CONSORT-AI flowchart of participant recruitment.

Power analysis was based on the secondary hypothesis testing (ie, determining the effects of Eleos versus TAU on PHQ-9 and GAD-7 at the 2-month assessment). Based on pilot data [1], we anticipated a large effect size (Cohen d=0.8) at the 2-month assessment for both depression and anxiety. A minimum of 23 individuals were necessary for each intervention arm, according to a 2-sided test with a P value of .05. Baseline group differences at randomization were assessed with independent, 2-tailed t tests. To assess changes in depressive and anxiety symptoms over the first 2 months of therapy, we used mixed effects models for each dependent variable. In both models, the fixed explanatory variables were intervention arm and baseline values. The models also contained random effects for time. Mixed effects modeling was performed by intent-to-treat analysis using all participants who provided more than one data point (eg, [23]). Missing data points were treated as missing at random [24]. All statistical analyses were performed on SPSS statistical software (version 27; IBM Corp).

A total of 72 adults who had been recommended outpatient CBT for a main diagnosis of a depressive or anxiety disorder were offered to participate in this study, of whom 47 (65%) provided their consent. In each group, 16 participants provided their clinical outcomes (70% and 67% of the 23 and 24 Eleos and TAU group participants, respectively). A total of 6 therapists participated in the AI group, and they treated between 1 to 6 clients; 8 therapists provided interventions in the TAU group, treating between 1 to 4 clients. Therapists could choose which study arm they wanted to join, and there were no significant group differences in the therapists’ profession, years of practice and training, or licensing status.

Participants in the AI group attended, on average, 67% more sessions than the TAU group, that is, 5.24 (SD 2.31, range 1-9) sessions, compared to 3.14 (SD 1.99, range 1-8) sessions during the first 2 months of therapy. All meetings were held in person.

Baseline PHQ-9 scores ranged between 3 and 16, and baseline GAD-7 scores ranged between 3 and 14. No significant group differences were found at baseline in depression (t=0.34; P=.86) and anxiety symptoms (t=0.37; P=.60). Patient-reported outcomes are outlined in Table 2. Patients in both groups endorsed reductions in depression and anxiety symptoms; however, a greater symptom reduction was found among patients whose therapist used the AI platform (Figures 4 and 5). Depression symptoms were reduced by 34% with Eleos, compared to 20% for TAU, with a large effect size for the AI group (d=0.82) versus a small effect size for the TAU group (d=0.34). Anxiety symptoms were reduced by 29% with Eleos, compared to 8% for TAU, with a larger effect size for Eleos (d=0.78) versus a small effect size for the TAU group (d=0.14). One participant in the TAU group was hospitalized during the study. No additional adverse events were recorded.

Patients in both groups reported high satisfaction with therapy and perceived it as highly helpful for them in achieving their goals, and no significant group differences were found; the average satisfaction rating was 4.50 (SD 0.50) in the AI group and 4.43 (SD 0.73) in the TAU group (t=0.20; P=.23); the mean perceived helpfulness of therapy was 4.25 (SD 0.66) in the AI group, compared to 4.57 (SD 0.73) in the TAU group (t=0.83; P=.81). None of the therapists or clients withdrew from the study. At the conclusion of the trial, when asked about their experience of the platform, therapists in the AI group provided positive qualitative reviews of both their augmented understanding of how they practice as well as the reduced documentation time, including the following statements: “It is very interesting to see the breakdown of CBT skills and who spoke more, etc. All the data that the program collects is fascinating”; “I believe it made me more aware of my use of CBT”; “I had some success with my documentation being more efficient”; and “It allowed for more time to engage with the client rather than being on the computer working on documentation.”

Therapists in the AI group submitted the progress notes for their sessions after 14 (SD 38) hours, on average, while the average submission time of the progress notes for the TAU sessions was 69 (SD 73) hours. The progress notes in both groups included required documentation aspects, such as the treatment plan, EBPs used in the session, purpose of the intervention, the client’s response, their progress toward treatment plan goals, and the plan for the next session, and they had less than 2 grammatical mistakes, on average. The notes submitted by the therapists in the AI group were, on average, 263 (SD 83) words long, while the notes submitted by the therapists in the TAU group were 318 (SD 159) words long. Table 3 presents the t tests and effect sizes for the progress notes’ between-group differences.

This study is the first RCT to report the feasibility, acceptability, and efficacy of an AI-powered platform designed to improve behavioral therapy through data and therapist feedback. With community-based clinics facing longer waitlists of patients presenting with more serious mental health difficulties, this study demonstrates the potential of digital tools to improve access to evidence-based, therapist-led care [25]. The study found that patient attendance in treatment supported by the AI platform was 2 times higher, and improvement in symptoms was 3 to 4 times better in these patients, compared to patients receiving TAU. Findings also indicate high treatment satisfaction and perceived helpfulness of therapy in both the AI and TAU groups. Additionally, the therapists in the AI group submitted the progress notes for their sessions with the study participants 55 hours earlier than therapists in the TAU group (14 hours vs 69 hours, respectively), and within the 24 hours from the service delivery time, as expected in routine care. Both groups’ notes contained the required documentation aspects and had similar, low rates of grammatical errors. However, notes from the AI group were more succinct, potentially simplifying communication with other providers.

Since this study was conducted in a routine behavioral health setting and dovetailed the interventions, procedures, and practices carried out in the clinic, the findings suggest that providing ongoing and timely feedback to therapists through an AI platform improves the quality of behavioral care provided in the field, resulting in meaningful changes in both clinical outcomes (patient retention and psychiatric symptoms) and effect sizes.

The results of this RCT suggest that providing therapy-specific AI-derived insights, such as a summary of patient’s concerns described in treatment, routine outcomes, and rate of EBPs provided by the therapist can accelerate the effects of behavioral therapy provided in a community-based clinic. Therapists in both study groups worked in the same community-based clinic and received similar training and ongoing supervision. Hence, findings indicate that community therapists can leverage technology to optimize the behavioral health services they provide. Currently, there are fewer therapists available to treat an increasing number of patients with mental health problems [26,27]. This situation is just one aspect of the overall community mental health issue. Another concern is the diminishing number of supervisors who can offer supervision to therapists with provisional licenses [28]. The AI’s ability to analyze session data and provide feedback to therapists and their supervisors is particularly noteworthy, as it allows for a more efficient use of time and expertise, and the potential for more data-informed and evidence-based care. These results demonstrate the potential for AI-powered platforms to be a key tool in addressing the growing mental health crisis, by accelerating the delivery of effective, evidence-based care and reducing the burden on community-based clinics.

This study included 47 participants and followed clients over a 2-month period. The therapy sessions provided by the therapists in the TAU group were not captured by the AI platform, and therefore, it is impossible to assess whether these therapists used other decision-support tools to improve practice. The small sample size and the length of the study are important limitations of this trial, particularly compared to other studies involving AI or digital tools. To further clarify these results, we encourage future research to replicate this study with a larger sample size and a longer follow-up period. Of note, the study participants presented with baseline depression and anxiety symptoms that were mostly mild to moderate in severity. Thus, future research should explore the applicability and effectiveness of AI platforms in treating individuals with severe mental illnesses and symptoms. This trial also lacks detailed data on the patients’ experience in therapy, as their satisfaction was assessed by only 2 items. The validity of AI feedback as an adjunct to clinical supervision was outside the scope of this study; however, it should be tested in future research.

The findings of this RCT suggest that providing therapy in behavioral health settings with the support of an AI platform was more effective than TAU. Using the AI platform led to greater session attendance and better depressive and anxiety outcomes. Results suggest that an AI platform designed to improve EBP improves client retention and treatment outcomes in real-world settings.