Preclinical Dialogue Simulation: Evaluating Response Accessibility in Conversational Artificial Intelligence for...

Area of Science:

• Natural Language Processing
• Clinical Linguistics
• Artificial Intelligence in Healthcare

Background:

• Conversational agents driven by large language models (LLMs) are being explored for clinical applications.
• Systematic evaluation methods for LLM behavior in speech-based therapeutic interactions with impaired speech are limited.
• Response Elaboration Training (RET) is a key therapeutic interaction for aphasia.

Purpose of the Study:

• To extend the Agent-Based Conversational Dialogue (ABCD) simulation method as a preclinical testbed.
• To evaluate how LLMs generate accessible clinician language when responding to simulated aphasic speech during RET.
• To benchmark different LLM families (Claude, GPT, Gemini) and configurations.

Main Methods:

• Simulated multi-turn spoken therapeutic dialogues between an LLM-clinician and an AI-simulated aphasic patient using the ABCD framework.
• Controlled manipulation of impairment profiles, prompting strategies (zero-shot, few-shot), and reasoning modes (standard, advanced).
• Quantification of response accessibility using established readability metrics and a composite score.

Main Results:

• Distinct accessibility patterns emerged across LLM architectures and configurations.
• Few-shot prompting and advanced reasoning generally improved response accessibility.
• Gemini exhibited superior accessibility under zero-shot, standard reasoning conditions.

Conclusions:

• LLMs demonstrate systematic differences in adapting clinician language for impaired speech.
• The ABCD framework provides a scalable, preclinical simulation for benchmarking conversational AI in clinical dialogue.
• Findings offer guidance for selecting and configuring LLMs prior to clinical deployment in communication rehabilitation.