Randomized Trial of a Generative AI Chatbot for Mental Health Treatment

Generative artificial intelligence (Gen-AI) chatbots hold promise for building highly personalized, effective mental health treatments at scale, while also addressing user engagement and retention issues common among digital therapeutics. We present a randomized controlled trial (RCT) testing an expert-fine-tuned Gen-AI-powered chatbot, Therabot, for mental health treatment.

We conducted a national, randomized controlled trial of adults (N=210) with clinically significant symptoms of major depressive disorder (MDD), generalized anxiety disorder (GAD), or at clinically high risk for feeding and eating disorders (CHR-FED). Participants were randomly assigned to a 4-week Therabot intervention (N=106) or waitlist control (WLC; N=104). WLC participants received no app access during the study period but gained access after its conclusion (8 weeks). Participants were stratified into one of three groups based on mental health screening results: those with clinically significant symptoms of MDD, GAD, or CHR-FED. Primary outcomes were symptom changes from baseline to postintervention (4 weeks) and to follow-up (8 weeks). Secondary outcomes included user engagement, acceptability, and therapeutic alliance (i.e., the collaborative patient and therapist relationship). Cumulative-link mixed models examined differential changes. Cohen's d effect sizes were unbounded and calculated based on the log-odds ratio, representing differential change between groups.

Therabot users showed significantly greater reductions in symptoms of MDD (mean changes: -6.13 [standard deviation {SD}=6.12] vs. -2.63 [6.03] at 4 weeks; -7.93 [5.97] vs. -4.22 [5.94] at 8 weeks; d=0.845-0.903), GAD (mean changes: -2.32 [3.55] vs. -0.13 [4.00] at 4 weeks; -3.18 [3.59] vs. -1.11 [4.00] at 8 weeks; d=0.794-0.840), and CHR-FED (mean changes: -9.83 [14.37] vs. -1.66 [14.29] at 4 weeks; -10.23 [14.70] vs. -3.70 [14.65] at 8 weeks; d=0.627-0.819) relative to controls at postintervention and follow-up. Therabot was well utilized (average use >6 hours), and participants rated the therapeutic alliance as comparable to that of human therapists.

This is the first RCT demonstrating the effectiveness of a fully Gen-AI therapy chatbot for treating clinical-level mental health symptoms. The results were promising for MDD, GAD, and CHR-FED symptoms. Therabot was well utilized and received high user ratings. Fine-tuned Gen-AI chatbots offer a feasible approach to delivering personalized mental health interventions at scale, although further research with larger clinical samples is needed to confirm their effectiveness and generalizability. (Funded by Dartmouth College; ClinicalTrials.gov number, NCT06013137.).