Related Experiment Video
Updated: Jan 8, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Illustrating a framework for assessing generative artificial intelligence-based conversational agents for mental
C Desage1, T Patterson1, Y Wu1
1Palo Alto University, Children and Adolescents Psychotherapy and Technology Lab, Palo Alto, CA, USA.
Background:
Generative Artificial Intelligence (AI)-based conversational agents (CAs) hold promise for expanding mental health care access.
Aims:
This study aimed to illustrate the implementation of the Thera Turing Test by comparing the quality of conversations delivered by a human therapist and a CA.
Methods:
The study examined two conversations of Parent Management Training, Psychoeducation, and Special Time, delivered by an AI CA (Pat) utilizing written online-conversation. All conversations were evaluated by graduate psychology students using the Thera Turing Test framework. Treatment fidelity and common therapeutic factors were assessed.
Results:
For both conversations (psychoeducation and special time), both the human therapist and Pat yielded high treatment fidelity (94.44% and 86% respectively). For common factors, the human therapist yielded a score of 90.80%, which was higher than all the Pat conversations.
Discussion:
These preliminary findings indicate that when comparing treatment fidelity between sessions conducted by a human and sessions conducted by Pat, both met a high treatment fidelity at comparable levels. Regarding common factors, Pat received lower ratings than the human therapist, indicating that there is room for improvement. Overall, this report illustrates the benefits of using the Thera Turing Test to assess the quality of CAs and inform improvements needed.
Related Concept Videos
Cognitive Therapy
Non-equilibrium in the Cell
Automatic Processing and Automatic Social Behavior
Rational Emotive Behavior Therapy
SBAR II: Application of SBAR
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
Treatment Strategies for Psychological Disorders
Psychological therapies focus on modifying emotions, thoughts, and behaviors through talking, interpreting, listening, rewarding, challenging, and modeling. Clinical psychologists, counselors, and social workers commonly practice psychotherapy. Clinical...