Related Experiment Video
Updated: Sep 11, 2025
06:42
Continuous Theta Burst Stimulation of the Posterior Medial Frontal Cortex to Experimentally Reduce Ideological Threat Responses
Published on: September 28, 2018
11.7K
Evaluating the Efficacy of ChatGPT-3.5 Versus Human-Delivered Text-Based Cognitive-Behavioral Therapy: A Comparative
Sebastian Acevedo1,2, Esha Aneja3, Douglas J Opler2
1Department of Psychiatry, Emory University School of Medicine, Atlanta.
American Journal of Psychotherapy
|August 18, 2025
Summary
Human therapists significantly outperformed AI chatbots like ChatGPT-3.5 in delivering text-based cognitive-behavioral therapy (CBT). While AI may assist, current implementations lack the personalized depth for standalone mental health treatment.
Area of Science:
- Mental Health
- Artificial Intelligence
- Digital Therapeutics
Background:
- The integration of artificial intelligence (AI) into mental healthcare offers potential for increased accessibility and novel therapeutic approaches.
- Text-based cognitive-behavioral therapy (CBT) is a widely recognized and effective form of psychotherapy.
- Evaluating the efficacy of AI in delivering therapeutic interventions is crucial for responsible implementation.
Purpose of the Study:
- To compare the effectiveness of an AI therapist (ChatGPT-3.5) against a human therapist in delivering text-based CBT.
- To assess the capabilities and limitations of current AI in a therapeutic context using established metrics and qualitative feedback.
Main Methods:
- A cross-sectional survey was conducted with 75 mental health professionals and trainees.
- Participants evaluated anonymized text-based CBT transcripts from both an AI therapist and a human therapist using the Cognitive Therapy Rating Scale (CTRS).
- Qualitative feedback was collected to further assess therapist performance and patient experience.
Main Results:
- Human therapists significantly outperformed ChatGPT-3.5 across most CTRS domains (p<0.001).
- Respondents rated human therapists higher for agenda setting (52% vs. 28%) and overall effectiveness (29% vs. 9%).
- While AI showed similar understanding of internal reality (36% vs. 19%), it was perceived as less personalized and more rigid than human therapists.
Conclusions:
- Current AI implementations, such as ChatGPT-3.5, show potential as complementary tools but lack the depth for independent use in mental health therapy.
- Findings are specific to this AI model and deployment, and cannot be generalized to all AI-based therapy due to diverse delivery methods.
- Further research is needed to explore the nuanced capabilities and limitations of AI across various therapeutic modalities.
Related Concept Videos
01:25Cognitive Therapy
269
Cognitive therapy, pioneered by Aaron T. Beck in the 1960s, is a structured approach to addressing psychological distress by focusing on the influence of thoughts on emotions and behaviors. All cognitive therapies involve the basic assumption that human beings have control over their feelings, and that how individuals feel about something depends on how they think about it. Unlike psychoanalytic methods that delve into unconscious processes or humanistic approaches emphasizing...
269
01:24Rational Emotive Behavior Therapy
111
Cognitive-behavioral therapies (CBTs) are grounded in the belief that our thoughts profoundly influence our emotions and actions. Advocates of CBT emphasize three core assumptions: first, that cognitions are identifiable and measurable; second, that they are central to psychological functioning; and third, that irrational or maladaptive beliefs can be replaced with rational and adaptive ones. This transformative approach to therapy has paved the way for specific models such as Albert...
111
01:26Group Therapy
85
Group therapy is a sociocultural approach to psychological treatment, where individuals with shared psychological challenges come together under the guidance of a mental health professional. This therapeutic modality offers unique opportunities for individuals to connect, share, and grow within the context of a supportive group. By fostering mutual understanding and collaboration, group therapy can address a range of psychological concerns effectively, often complementing or surpassing the...
85
01:25Interpersonal Psychotherapy
99
Interpersonal psychotherapy (IPT) is a structured, time-limited therapeutic approach initially developed to treat depression. It integrates key concepts from psychodynamic, humanistic, and cognitive-behavioral therapies, making it a uniquely eclectic framework. The therapy is rooted in the interpersonal theories of Adolph Meyer and Harry Stack Sullivan, as well as John Bowlby's attachment theory, and focuses on the interplay between interpersonal relationships and emotional well-being.
99
01:30Electroconvulsive Therapy
185
Electroconvulsive therapy (ECT), or shock therapy, remains a critical biomedical intervention for severe, treatment-resistant depression. While its origins can be traced back to Hippocrates' observations that malaria-induced convulsions alleviated mental illness, modern ECT has evolved significantly from its earlier, more primitive applications. First introduced in 1938 by Ugo Cerletti and his colleagues, ECT involves inducing controlled seizures using electrical currents. In its early...
185
01:24Treatment Strategies for Psychological Disorders
267
Treatment approaches for psychological disorders fall into three main categories: psychological, biological, and sociocultural. Each approach targets different aspects of mental health, requiring varying levels of education and training.
Psychological therapies focus on modifying emotions, thoughts, and behaviors through talking, interpreting, listening, rewarding, challenging, and modeling. Clinical psychologists, counselors, and social workers commonly practice psychotherapy. Clinical...
Psychological therapies focus on modifying emotions, thoughts, and behaviors through talking, interpreting, listening, rewarding, challenging, and modeling. Clinical psychologists, counselors, and social workers commonly practice psychotherapy. Clinical...
267