Spectral and Temporal Envelope Cues for Human and Automatic Speech Recognition in Noise

Area of Science:

• Speech processing
• Auditory perception
• Human-computer interaction

Background:

• Speech recognition relies on acoustic features, including spectral and temporal cues.
• Human listeners prioritize temporal envelope, while automated speech recognition (ASR) emphasizes spectral analysis.
• Understanding how noise impacts human and ASR performance is crucial for developing robust systems.

Purpose of the Study:

• To compare sentence recognition scores between humans and ASR (Dragon) under manipulated spectral and temporal cues in various noise conditions.
• To investigate the differential effects of noise types and spectral resolution on human and ASR performance.
• To elucidate the distinct listening strategies employed by humans and ASR in noisy environments.

Main Methods:

• Speech recognition tests were conducted with humans and Dragon ASR software.
• Temporal fine structure was degraded using noise or tone vocoders.
• Spectral information was altered by varying the number of frequency channels.
• Three noise types were used: white noise, time-reversed multi-talker noise, and fake-formant noise.

Main Results:

• White noise more disruptive than fake-formant noise for humans at 20 dB SNR with 4 vocoding channels.
• ASR struggled significantly with 4 vocoding channels even at 20 dB SNR.
• Fake-formant noise most severely impacted ASR, while white noise affected speech segmentation.
• Increasing spectral resolution yielded non-monotonic ASR behavior with white noise.
• ASR performance improved with tone vocoders.

Conclusions:

• Human listeners and ASR systems exhibit fundamentally different processing strategies in noisy conditions.
• Temporal envelope cues are critical for human speech recognition, whereas spectral cues are more important for ASR.
• Noise characteristics significantly influence the performance of both humans and ASR, but in distinct ways.
• Fake-formant noise disrupts ASR by interfering with spectral cues, while white noise impacts speech segmentation.