Does Recording Hardware Matter for Clinical Speech Recognition? Evaluating ASR Performance Across Consumer Devices

Area of Science:

• Medical Informatics
• Speech Technology
• Clinical Documentation

Background:

• Ambient clinical intelligence (ACI) systems leverage automatic speech recognition (ASR) to transcribe patient-provider conversations for clinical documentation.
• Current ASR evaluations often occur under controlled settings with specialized equipment, potentially not reflecting real-world usage.

Purpose of the Study:

• To assess the impact of various recording devices on the transcription performance of contemporary ASR engines in a clinical dialogue context.
• To compare the influence of recording hardware versus ASR engine choice on transcription accuracy.

Main Methods:

• Thirty-five primary care encounters were re-enacted and simultaneously recorded using five distinct devices: smartphone, laptop microphone, portable recorder, clip-on microphone, and desktop microphone.
• Six different ASR engines were evaluated using word error rate (WER), clinical concept extraction (precision and recall), and sentence-level semantic similarity metrics.

Main Results:

• Median word error rates (WER) across ASR engines ranged from 16.7% to 20.7%.
• While recording device differences were statistically significant, the choice of ASR engine demonstrated a greater impact on transcription performance.
• ASR engines showed considerable robustness when used with consumer-grade recording hardware.

Conclusions:

• Contemporary ASR engines exhibit resilience to variations in consumer-grade recording devices for clinical applications.
• ASR model selection appears to be a more critical factor than recording device configuration for optimizing transcription performance in real-world ACI deployments.