Predicting Symptom Change in Mental Health Chatbot Interventions: A Meta-Analysis

Mental health-focused chatbots have emerged as scalable interventions that deliver therapeutic content through conversational interfaces. This meta-analysis of 35 studies (N = 1,780) indicates that chatbots can reduce depression (g = -.53, 95% CI [-.67,-.40]) and anxiety (g = -.36, 95% CI [-.46,-.27]) symptoms over time. However, wide prediction intervals reveal substantial heterogeneity, suggesting that while some interventions are associated with symptom reduction, others may increase them. Moderator analyses revealed that therapeutic alliance significantly predicted reductions in both depression and anxiety symptoms. Task and goal alliance significantly predicted a reduction in depression symptoms, while bond alliance did not. Similarly, treatment acceptability significantly moderated depression symptom change, noting the importance of perceived intervention appropriateness. Mental health chatbots designed using cognitive behavioral therapy and interventions delivered over longer durations were associated with larger symptom reductions, particularly for anxiety. In addition, populations with both clinical and self-reported diagnoses were associated with larger reductions in depression symptoms compared to interventions recruiting general populations. Findings also reveal considerable gaps in mental health chatbot development and evaluation, as few studies have examined key communication-related moderators, such as therapeutic alliance and acceptability, despite their demonstrated importance in the mental health communication and therapeutic literature.