Conversational Agent Interventions for Mental Health Problems: Systematic Review and Meta-analysis of Randomized Controlled Trials

Background Mental health problems are a crucial global public health concern. Owing to their cost-effectiveness and accessibility, conversational agent interventions (CAIs) are promising in the field of mental health care. Objective This study aims to present a thorough summary of the traits of CAIs available for a range of mental health problems, find evidence of efficacy, and analyze the statistically significant moderators of efficacy via a meta-analysis of randomized controlled trial. Methods Web-based databases (Embase, MEDLINE, PsycINFO, CINAHL, Web of Science, and Cochrane) were systematically searched dated from the establishment of the database to October 30, 2021, and updated to May 1, 2022. Randomized controlled trials comparing CAIs with any other type of control condition in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and positive and negative affect were considered eligible. This study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Data were extracted by 2 independent reviewers, checked by a third reviewer, and pooled using both random effect models and fixed effects models. Hedges g was chosen as the effect size. Results Of the 6900 identified records, a total of 32 studies were included, involving 6089 participants. CAIs showed statistically significant short-term effects compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38), generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36), specific anxiety symptoms (g=0.47, 95% CI 0.07-0.86), quality of life or well-being (g=0.27, 95% CI 0.16-0.39), general distress (g=0.33, 95% CI 0.20-0.45), stress (g=0.24, 95% CI 0.08-0.41), mental disorder symptoms (g=0.36, 95% CI 0.17-0.54), psychosomatic disease symptoms (g=0.62, 95% CI 0.14-1.11), and negative affect (g=0.28, 95% CI 0.05-0.51). However, the long-term effects of CAIs for the most mental health outcomes were not statistically significant (g=−0.04 to 0.39). Personalization and empathic response were 2 critical facilitators of efficacy. The longer duration of interaction with conversational agents was associated with the larger pooled effect sizes. Conclusions The findings show that CAIs are research-proven interventions that ought to be implemented more widely in mental health care. CAIs are effective and easily acceptable for those with mental health problems. The clinical application of this novel digital technology will conserve human health resources and optimize the allocation of mental health services. Trial Registration PROSPERO CRD42022350130; https://tinyurl.com/mvhk6w9p


Methods
Search strategy 7 Present the full search strategies for all databases, registers and websites, including any filters and limits used. Methods Selection process 8 Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.

Methods
Data collection process 9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.

Methods
Data items 10a List and define all outcomes for which data were sought. Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.

Methods
10b List and define all other variables for which data were sought (e.g. participant and intervention characteristics, funding sources). Describe any assumptions made about any missing or unclear information.

Methods
Study risk of bias assessment 11 Specify the methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process.

Effect measures
12 Specify for each outcome the effect measure(s) (e.g. risk ratio, mean difference) used in the synthesis or presentation of results.

Synthesis methods
13a Describe the processes used to decide which studies were eligible for each synthesis (e.g. tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)).

Section and Topic
Item # Checklist item Location where item is reported 13b Describe any methods required to prepare the data for presentation or synthesis, such as handling of missing summary statistics, or data conversions.

Methods
13c Describe any methods used to tabulate or visually display results of individual studies and syntheses. Methods 13d Describe any methods used to synthesize results and provide a rationale for the choice(s). If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used.

Methods
13e Describe any methods used to explore possible causes of heterogeneity among study results (e.g. subgroup analysis, meta-regression).

Methods
13f Describe any sensitivity analyses conducted to assess robustness of the synthesized results. Methods Reporting bias assessment 14 Describe any methods used to assess risk of bias due to missing results in a synthesis (arising from reporting biases). Methods

Certainty assessment
15 Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome. Methods

Study selection
16a Describe the results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.

Results
16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded.

Study characteristics
17 Cite each included study and present its characteristics. Results

Risk of bias in studies
18 Present assessments of risk of bias for each included study. Results

Results of individual studies
19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g. confidence/credible interval), ideally using structured tables or plots.

Results of syntheses
20a For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies. Results 20b Present results of all statistical syntheses conducted. If meta-analysis was done, present for each the summary estimate and its precision (e.g. confidence/credible interval) and measures of statistical heterogeneity. If comparing groups, describe the direction of the effect.

Results
20c Present results of all investigations of possible causes of heterogeneity among study results. Results 20d Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results.
Results Reporting 21 Present assessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed. Results

Section and Topic
Item # Checklist item Location where item is reported biases Certainty of evidence 22 Present assessments of certainty (or confidence) in the body of evidence for each outcome assessed. Results

Discussion
23a Provide a general interpretation of the results in the context of other evidence. Discussion 23b Discuss any limitations of the evidence included in the review. Discussion 23c Discuss any limitations of the review processes used.
Discussion 23d Discuss implications of the results for practice, policy, and future research. Discussion OTHER INFORMATION Registration and protocol 24a Provide registration information for the review, including register name and registration number, or state that the review was not registered.
Methods 24b Indicate where the review protocol can be accessed, or state that a protocol was not prepared. Methods 24c Describe and explain any amendments to information provided at registration or in the protocol.
Methods Support 25 Describe sources of financial or non-financial support for the review, and the role of the funders or sponsors in the review. Acknowledgements Competing interests 26 Declare any competing interests of review authors.

Conflicts of Interest
Availability of data, code and other materials 27 Report which of the following are publicly available and where they can be found: template data collection forms; data extracted from included studies; data used for all analyses; analytic code; any other materials used in the review.
Data availability  Figure S1. Risk of bias summary

Figure S2. Risk of bias graph
Among the included studies, 1 was assessed to have a high risk of bias due to poor randomization, 2 had a high risk of bias due to the lack of allocation concealment, 20 were assessed high on performance bias, 3 were assessed high on detection bias, 1 was assessed high on attrition bias, and 5 were assessed high on reporting bias. The interrater reliability suggested substantial agreement between the raters on each criterion of the Cochrane Collaboration's tool (Cohen's kappa = 0.86, 0.79, 0.67, 0.89, 0.93, 0.75 and 0.83, respectively).