A Comparison Between Clinical Guidelines and Real-World Treatment Data in Examining the Use of Session Summaries:...

Area of Science:

• Mental Health Care Delivery
• Behavioral Interventions
• Psychological Treatment Fidelity

Background:

• Behavioral interventions are effective in clinical trials but often suffer from suboptimal treatment fidelity in real-world settings.
• Therapist drift, or deviation from evidence-based protocols, reduces treatment effectiveness.
• Assessing adherence to treatment protocols outside of large randomized controlled trials has been historically limited.

Purpose of the Study:

• To investigate the utilization of a standard behavioral treatment component: prompting clients to summarize their therapy sessions.
• To understand how this summarization practice consolidates client understanding and treatment plan adherence.
• To explore the frequency and context of session summaries in routine mental health care.

Main Methods:

• Analysis of 17,607 behavioral therapy sessions across 37 US programs using a therapy-specific artificial intelligence (AI) platform.
• Automatic speech recognition transcribed sessions, differentiating therapist and client utterances.
• Two psychologists validated identified session summary prompts within the transcribed data.

Main Results:

• Only 54 (0.30%) of sessions included a client summary, despite clinical recommendations.
• Summaries primarily focused on relationships, work, change, and alcohol use.
• Sessions with summaries exhibited increased therapist interventions, including validation, complex reflections, and problem-solving.

Conclusions:

• This study is the first to evaluate real-world treatment practices using a large, diverse dataset.
• Findings confirm that maintaining fidelity to core components of empirically supported psychological interventions is challenging in practice.
• Results can guide the development of AI tools to provide feedback, enhance evidence-based practice delivery, and improve mental health care outcomes.