Preclinical Dialogue Simulation: Evaluating Response Accessibility in Conversational Artificial Intelligence for...

Large language model (LLM)-driven conversational agents are increasingly considered for use in clinical contexts, yet systematic approaches for evaluating their behavior in impairment-rich, speech-based therapeutic interactions remain limited. This study extends the Agent-Based Conversational Dialogue (ABCD) simulation method as a preclinical testbed to evaluate how LLMs generate accessible clinician language when responding to characteristic aphasic speech during Response Elaboration Training.

ABCD was used to simulate multi-turn spoken therapeutic dialogues between an LLM-driven clinician and an artificial intelligence (AI)-simulated aphasic patient, enabling controlled manipulation of impairment profiles, prompting strategies, and reasoning modes without human participants. Three LLM families (Claude, GPT, and Gemini) were benchmarked under zero-shot and few-shot prompting and standard versus advanced reasoning. Response accessibility was quantified using established readability metrics (Flesch Reading Ease, Dale-Chall) and a composite score derived from 16 standardized readability measures.

Distinct accessibility signatures emerged across model architectures and configurations. Few-shot prompting and advanced reasoning generally yielded more accessible clinician responses, whereas Gemini demonstrated superior accessibility under zero-shot, standard reasoning.

LLMs differ systematically in their capacity to adapt clinician language to impaired speech. ABCD provides a scalable, preclinical dialogue simulation framework for benchmarking conversational AI in clinically oriented, impairment-rich dialogue across multidimensional constraints. It offers guidance for model selection and configuration prior to clinical translation in communication rehabilitation.

https://doi.org/10.23641/asha.32736078.