AI-generated clinical summaries: errors and susceptibility to speech and speaker variability

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Natural Language Processing

Background:

• Clinical AI Scribes (CAIS) aim to automate medical documentation.
• Variability in patient communication (personality, accents, speech impairments) may impact CAIS accuracy.
• Ensuring CAIS reliability across diverse patient populations is crucial for safe deployment.

Purpose of the Study:

• To assess the impact of patient communication variability on Clinical AI Scribe (CAIS) accuracy.
• To identify specific factors (personality, accents, speech impairments) that degrade CAIS performance.
• To inform validation and monitoring strategies for CAIS pre-deployment.

Main Methods:

• Simulated primary care consultations using trained actors.
• Enacted scenarios with diverse patient personality types.
• Generated accent variations using consultation transcripts and public speech impairment recordings.
• Classified CAIS errors (omissions, factual inaccuracies, hallucinations) against human transcripts.

Main Results:

• No statistically significant differences in CAIS errors were found across personality types (p>0.05).
• CAIS performance was stable across patient and doctor accents, with non-significant differences (p>0.851 for patients, p>0.980 for doctors).
• Phonological impairment significantly reduced CAIS recognition accuracy (p<0.001), while cleft palate and vowel disorders had minimal impact.

Conclusions:

• CAIS demonstrates broad stability across communication styles and most accents under controlled conditions.
• Vulnerability to specific speech characteristics, notably phonological impairment, requires attention.
• Operational deployment necessitates clinician-in-the-loop verification, subgroup performance monitoring, and 'switch-off' criteria for severe phonological patterns.