Comparing human and machine speech recognition in noise with QuickSIN.

Area of Science:

• Speech processing
• Human-computer interaction
• Auditory perception

Background:

• Speech recognition systems are crucial for human-computer interaction.
• Evaluating speech recognition in noise is essential for real-world applications.
• Human performance in noise provides a benchmark for system capabilities.

Purpose of the Study:

• To propose a novel test for characterizing automatic speech recognition (ASR) system performance in noise.
• To benchmark modern ASR systems against human performance using the QuickSIN test.
• To establish a standardized metric for evaluating speech-in-noise recognition abilities of ASR.

Main Methods:

• Utilized the QuickSIN (Quick Speech in Noise) test, commonly used in audiology.
• Measured the signal-to-noise ratio (SNR) at which ASR systems achieve 50% keyword recognition.
• Compared ASR performance in noise to established human performance data.

Main Results:

• Modern ASR systems, trained on extensive unsupervised data, were evaluated.
• ASR performance in noise varied, with some systems performing at a normal human level.
• Other systems demonstrated mild impairment in noisy conditions compared to human participants.

Conclusions:

• The proposed test effectively characterizes ASR performance in challenging acoustic environments.
• Modern ASR systems exhibit human-like variability in speech recognition accuracy under noisy conditions.
• Grounding ASR performance metrics to human abilities is vital for developing robust speech technologies.