This study was not preregistered. This systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (Table S1 in Multimedia Appendix 1) [30]. To locate studies assessing the effectiveness of AI-driven CAs for mental health problems in adolescents and young adults, two independent researchers (LYN and SXH) conducted a comprehensive search across five databases: PubMed, PsycINFO, EMBASE, Cochrane Library, and Web of Science. The search spanned from the inception of each database up to August 6, 2024. The following search terms were used: (robot OR social bot OR dialogue system OR conversational agent OR conversational bot OR conversational system OR conversational interface OR chatbot OR chat bot OR chatterbot OR chatter bot OR chat-bot OR smartbot OR smart bot OR smart-bot OR virtual coach OR virtual agent OR embodied agent OR relational agent OR avatar OR virtual character OR animated character OR virtual human OR virtual assistant OR digital assistant OR counseling agent) AND (mental illness OR mental disorder OR affective disorder OR psychotic disorder OR posttraumatic stress disorder OR PTSD OR distress OR depress OR anxiety OR bipolar OR schizophrenia OR psychosis OR mental health OR mental wellness OR wellbeing OR well-being OR SWB OR happiness OR happy OR positive affect OR negative affect OR positive emotion OR negative emotion OR mood OR life satisfaction OR healthy relationship OR resilience OR self-efficacy). Detailed Search strategies were also provided in Multimedia Appendix 2. No filters were applied to ensure the inclusion of all relevant studies. In addition, reference lists of included studies and previous reviews were manually searched to identify any further eligible studies. A detailed search strategy is provided in the Multimedia Appendix 2.

Studies were selected based on these criteria (Table 1):

For each included study, the following data were extracted: authorship, year of publication, participant characteristics (eg, sample size, gender distribution, and mean age), CA specifications (eg, name, platform, and interaction mode), intervention details (eg, length and control group type), and measures. Methodological quality was assessed using the Cochrane Risk of Bias tool [32], considering factors such as random sequence generation, allocation concealment, blinding of participants and assessors, handling of incomplete outcome data, and selective reporting. In total, 2 authors (WW and XS) independently conducted the data extraction and quality assessments. Disagreements were resolved through discussion, with the involvement of a third author (YH) when necessary. The results of the risk of bias assessments are visually presented in a summary graph.

For each study, the means, SDs, and sample sizes at posttest were extracted to compute ESs. As very few studies included a follow-up measurement and the follow-up intervals differed, follow-up ESs were not calculated. When multiple studies reported data for the same outcome, pooled ESs were calculated. Given the small sample sizes in some studies, Hedges g was used to adjust for bias [33]. All ESs were coded so that positive Hedges g values indicated superior outcomes for the treatment group relative to the control group. Where intention-to-treat and completer analyses were both available, data from the intention-to-treat analyses were used. Follow-up data were not analyzed due to insufficient reporting across studies. ES calculations and overall estimates were performed using Comprehensive Meta-Analysis software version 3.0 (Biostat, Inc) and Stata SE version 15.1 (StataCorp). For studies that did not report means and SDs, alternative statistics (eg, Cohen d, t values, F values) were used. Multiarm trials were treated in accordance with Cochrane guidelines by combining means and SDs to create a single pairwise comparison [34]. Given the expected heterogeneity across studies, a random-effects model was applied to estimate the mean ESs [35]. Heterogeneity was examined using the Q statistic and I² index [33]. Outliers were identified as studies whose 95% CIs lay outside the 95% CI of the overall estimate [36]. A “leave-one-out” sensitivity analysis was conducted to assess the robustness of the results. Subgroup and meta-regression analyses were performed with outliers excluded to explore possible causes of heterogeneity among studies.

To assess publication bias, three methods were used: (1) visual inspection of funnel plots; (2) Duval and Tweedie’s trim-and-fill procedure [37] to adjust ES estimates for publication bias; and (3) Egger test for funnel plot asymmetry [38]. Moderator analyses were performed for both categorical and continuous moderators where heterogeneity was significant (P<.10 or I²>25%) and the overall ES was significant (P<.05). Subgroup analyses used a mixed-effects model, while meta-regression used unrestricted maximum likelihood estimation. Moderators included age, gender, intervention length, interaction mode, delivery platform, sample type, and control group type, selected based on previous meta-analytic studies [28,39]. All visualization was conducted by R (RStudio, PBC) version 4.3.1 and Review Manager version 5.3 (The Cochrane Collaboration).

The initial database search identified 14,907 potentially relevant articles, supplemented by 2 additional studies found through manual reference list checks. The total sample size is 14,909. After removing 7412 duplicates, 7497 unique articles remained for screening. Titles and abstracts were reviewed, resulting in the exclusion of 7100 records. Subsequently, 397 full-text articles were assessed for eligibility. The PRISMA flow diagram outlining this process is provided (Figure 1). Ultimately, 14 articles with 15 RCTs were included in the systematic review and meta-analysis.

The characteristics of the 15 included RCTs are presented below (Table S2 in Multimedia Appendix 2 [18-25,40-45]). The sample size ranged from 42 to 415 participants, with a total pooled sample size of 1901 across all studies. The meta-analysis incorporated studies conducted with clinical (n=1), subclinical (n=7), and nonclinical (n=8) populations. Overall, 12 studies used retrieval-based CAs, 3 used generative CAs, and 1 study used both retrieval-based and generative CAs. Regarding interaction modes, 13 studies used text-based CAs, and 3 used multimodal CAs.

The overall quality of the included studies was suboptimal. Only one study [41] satisfied all 6 quality criteria. In total, 3 studies met 4 criteria, while 5 studies met 3, and 6 studies met fewer than 3 criteria. Notably, a significant number of studies lacked sufficient information to assess certain criteria: 6 out of 15 studies did not report on the blinding of participants or personnel, 9 did not mention the blinding of outcome assessors, and 8 studies were not registered and lacked information on selective reporting. A summary of the risk of bias for each included randomized controlled trials examining the effect of AI-driven conversational agents in improving mental health among young people is displayed below (Figure 2).

This meta-analysis was the first comprehensive evaluation of the effectiveness of AI-driven CAs mental health intervention among young people. Overall, 16 studies with a total of 1974 participants were evaluated. Findings underscored the potential of AI-driven CAs to significantly alleviate depressive symptoms, particularly in subclinical populations. However, their effects on other mental health outcomes, such as anxiety, stress, and negative affect, were less robust, revealing important insights into both the promise and limitations of AI-driven interventions in this demographic.

The results of this meta-analysis revealed that AI-driven CAs demonstrated a moderate-to-large intervention effect on depressive symptoms. This finding aligns with previous research that has demonstrated the efficacy of AI-driven CAs in reducing depression among all age groups [28], which also revealed that AI-driven CAs had a moderate-to-large effect on depression. This suggests that AI-driven CAs, especially when enhanced with NLP and machine learning, can be particularly effective in mitigating depressive symptoms. To note, these results are more favorable compared with earlier meta-analyses that included non-NLP systems, which reported smaller ESs (Hedges g ranging from 0.26 to 0.29) for depression [39,46]. One possible explanation is that as AI-driven CAs can offer greater flexibility and adaptability in delivering therapeutic interventions [47,48], thus they are generally more effective in managing depressive symptoms than their non-NLP counterparts. It is also possible that the larger ES found in this study reflects that AI-based CAs are more beneficial to young people, for whom digital interventions may be more acceptable and engaging due to their familiarity with digital platforms [49]. This aligns with a previous review which indicated that younger age was associated with a larger effect of CAs on depressive symptoms [39].

In contrast to the substantial effects observed for depression, the effects of AI-driven CAs on anxiety, stress, positive affect, negative affect, and mental well-being in this age group were all nonsignificant. This aligns with previous meta-analyses that have found AI-based CA interventions to be less effective for anxiety, positive affect, negative affect, and psychological well-being compared to depression [28,50]. The nonsignificant findings for anxiety and stress in this meta-analysis may be explained by the limited inclusion of behavioral strategies, such as exposure therapy, in current AI-driven CAs. As anxiety and stress often require more intensive behavioral interventions [51], future iterations of AI-driven CAs may benefit from integrating these techniques to improve outcomes for anxiety-related symptoms. In addition, the small and nonsignificant effects on outcomes related to well-being may suggest that AI-based CAs were not yet able to enhance well-being in young people. It is possible that as most AI-driven CAs were based on CBT [47], they are less effective in cultivating positive psychological assets.

Subgroup analyses revealed the significant role of clinical versus subclinical type in moderating the efficacy of AI-driven CAs on depression intervention. Specifically, AI-driven CAs were particularly effective in subclinical populations. This finding aligns with a previous meta-analysis [28], suggesting that subclinical populations are more likely to benefit from AI-based CAs. This is consistent with the broader literature on psychological interventions, which has shown that these interventions are often more effective in promoting mental well-being for people with mental or physical health conditions compared to the general population [52]. Subclinical depression is clinically significant not only because it can cause considerable impairment requiring intervention, but also due to the heightened risk of progression to major depressive disorder, which can potentially be prevented with early treatment [53]. The notable intervention effectiveness of AI-driven CAs among young people with subclinical depression provides an important insight that these digital tools may serve as valuable early interventions to help mitigate the risk of developing more severe mental health conditions. In addition, heterogeneities within studies conducted in clinical and subclinical samples were all nonsignificant (P>.10), suggesting that the overall heterogeneity observed in the total sample may be attributed to differential effects of AI-driven CAs across clinical, subclinical, and nonclinical populations.

The nonsignificant moderating effects of interaction mode and delivery platform revealed that CA technical features may not influence the effectiveness of AI-driven CAs in reducing depression among young people. It is possible that as young people are familiar with digital platforms, they can interact effectively with AI-driven CAs with different technical features. Considering the diversity of AI-driven CAs in system orientation, topic constraints, and response patterns, they may meet the needs of different types of users, such as vertical AI-driven CAs (eg, Woebot) [19] where domain knowledge is more structured, intervention is more scientific, and compliant. Open content AI-driven CAs (eg, Replika) [20] can attract users’ attention and establish emotional connections through richer and more diverse dialogue content, making them suitable for early intervention in mental health (such as depression emotion recognition). AI-driven CAs with more structured response methods (eg, Manage Your Life Online) [21] can help individuals with poor self-management abilities to train and those users who are prone to contemplation may prefer to engage in open discussions with freely chatting CAs (eg, ChatGPT) [18]. Since AI-driven CAs can be used in diverse ways depending on user input, it remains unclear what participants were actually doing during these interventions, which also suggests that future research should further explore the impact of individual characteristics on the effectiveness of digital therapy. Age and sex also did not moderate posttest effect. This may reflect that AI-driven CAs could be effective for both males and females and young people in different age groups. In addition, intervention length did not moderate the ESs on depression, which may reflect that both short-course and long-course treatments delivered by AI-driven CAs could be effective in alleviating depressive symptoms. Finally, control group type, publication year, and study quality did not moderate posttest ESs, which further support the robustness of the effectiveness of AI-driven CAs for depression among young people.

Despite the promising results, several limitations should be acknowledged. First, the limited number of studies examining the long-term effects of AI-driven CAs prevented a thorough evaluation of the sustainability of treatment outcomes. As digital interventions continue to gain prominence, it is crucial for future studies to include follow-up assessments to better understand the durability of therapeutic effects. Second, the inclusion of only English-language studies may have introduced selection bias, limiting the generalizability of our findings. Third, some analyses had relatively lower statistical power due to the limited number of studies available for certain outcomes (eg, stress and mental well-being, both with N=4). Similarly, the number of studies in some subgroups was limited, which made it difficult to reach robust conclusions in subgroup analyses. Fourth, we included CAs based on various therapeutic orientations, which may lead to considerable heterogeneity in results. Finally, the current review did not address how users engage with AI-driven CAs. The variability in how people use AI-driven CAs introduces challenges in assessing treatment fidelity and determining whether specific features or therapeutic components of CAs drive effectiveness. Future research should explore user interaction themes and engagement patterns to better understand how AI-driven CAs function in practice and to identify key ingredients contributing to their clinical effectiveness.

Practically, the findings from this study support the integration of AI-driven CAs as part of mental health interventions targeting the early stages of depression. Their ability to adapt and tailor their interactions based on user inputs may make them more personalized and effective than early forms of digital mental health interventions. In addition, their accessibility makes them valuable tools, particularly for young people facing barriers to traditional therapy. For young people with depressive disorders, AI-driven CAs may be insufficient as a stand-alone treatment. Future research could explore their potential as an adjunct to traditional therapeutic approaches. To increase their adaptability and effectiveness, incorporating other evidence-based therapeutic approaches—such as adding exposure techniques for anxiety [52]—may enable CAs to better serve diverse mental health needs among young people. For instance, in vivo and imaginal exposure tasks have been successfully integrated into smartphone applications for treating anxiety and stress-related disorders [54], suggesting their potential for integration into AI-driven CAs as well.

With continued advancements in AI technologies, these digital tools have the potential to play a pivotal role in bridging the mental health treatment gap for young people. This meta-analysis provides robust evidence for the effectiveness of AI-driven CAs in reducing depressive symptoms among young people, particularly in subclinical populations. Their effectiveness for anxiety, stress, and outcomes related to well-being is not robust, highlighting the need for further development. Future research should focus on refining the therapeutic capabilities of AI-driven CAs and exploring long-term mental health outcomes.