Racial disparities in automated speech recognition.

Area of Science:

• Speech Technology
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Automated speech recognition (ASR) systems are increasingly prevalent, powering virtual assistants and dictation tools.
• Recent advancements in deep learning and large datasets have improved ASR quality.
• Concerns exist regarding equitable performance across diverse demographic groups.

Purpose of the Study:

• To evaluate the performance disparities of five leading ASR systems across racial groups.
• To identify the extent of performance differences in transcribing speech from Black and white individuals.

Main Methods:

• Assessed five state-of-the-art ASR systems (Amazon, Apple, Google, IBM, Microsoft).
• Utilized a corpus of 19.8 hours of structured interviews from 42 white and 73 Black speakers across five US cities.
• Matched audio data for age and gender to isolate the impact of race.

Main Results:

• All five ASR systems demonstrated significant racial disparities in transcription accuracy.
• Black speakers experienced an average word error rate (WER) of 0.35, compared to 0.19 for white speakers.
• Disparities were linked to acoustic models, evident even with identical phrases.

Conclusions:

• Current ASR systems exhibit substantial performance gaps based on race.
• Diverse training datasets, including African American Vernacular English, are crucial for reducing these disparities.
• Ensuring inclusive speech recognition technology requires addressing underlying model biases.