Related Experiment Video
Updated: Jun 2, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Evaluating the interpretability of clinical speech AI models: Lessons from two user studies
Lingfeng Xu1, Visar Berisha1,2, Julie Liss1
1College of Health Solutions, Arizona State University, 550 N 3rd Street, Phoenix, 85004, Arizona, USA.
Computer Speech & Language
|June 1, 2026
Summary
Current AI interpretability methods in speech pathology can mislead clinicians, causing them to misinterpret feature influence as clinical severity. New designs are needed for effective AI integration in clinical practice.
Area of Science:
- Clinical Speech Science
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Artificial intelligence (AI) interpretability is crucial for clinician trust and decision support in clinical speech applications.
- Existing methods like SHapley Additive exPlanations (SHAP) may not be suitable for speech-language pathology (SLP) due to unfamiliar data modalities and workflow misalignment.
Purpose of the Study:
- To systematically evaluate a common SHAP-based interpretation design for AI in dysarthria detection.
- To assess factors including faithfulness, efficiency, cognitive load, task performance, mental model, trust, understandability, and decision relevance.
Main Methods:
- Two user studies were conducted with speech-language pathology students.
- A bar chart visualizing acoustic feature influence on AI decisions was evaluated in the context of dysarthria detection.
Main Results:
- The SHAP bar chart design misled participants, causing them to equate feature influence with clinical severity.
- Participants experienced difficulties understanding AI mechanisms and found interpretations unable to address clinical questions.
- Discrepancies were observed between human and AI reasoning.
Conclusions:
- Current AI interpretation designs pose risks, potentially introducing bias and burden rather than clinical insight.
- There is a critical need for AI interpretation designs aligned with clinical reasoning patterns for effective integration into SLP practice.