Towards decoupling frontend enhancement and backend recognition in monaural robust ASR

Area of Science:

• Speech processing
• Machine learning
• Signal processing

Background:

• Speech enhancement (SE) algorithms can improve noisy speech intelligibility.
• Monaural SE has historically underperformed compared to direct noisy speech training for automatic speech recognition (ASR).
• A gap persists between SE advancements and robust ASR system development.

Purpose of the Study:

• To bridge the gap between SE and ASR by proposing novel SE models.
• To enable ASR systems trained solely on clean speech to perform effectively in noisy environments.
• To advance robust ASR through improved SE frontends.

Main Methods:

• Developed three SE models: Attentive Recurrent Network (ARN) in the time-domain, TF-CrossNet in the time-frequency domain, and MP-SENet based on magnitude-phase.
• Decoupled SE frontend from the ASR backend, training the ASR only on clean speech.
• Evaluated performance on WSJ, CHiME-2, LibriSpeech, and CHiME-4 corpora.

Main Results:

• ARN, TF-CrossNet, and MP-SENet significantly improved ASR performance in noisy and reverberant conditions.
• The proposed systems outperformed baseline ASR models trained directly on corrupted speech.
• Achieved state-of-the-art results on CHiME-2 (5.6% WER) and CHiME-4 (3.3/4.4% WER), generalizing to real acoustic scenarios.

Conclusions:

• The proposed SE models effectively eliminate the divide between SE and ASR.
• Decoupled SE frontends allow for robust ASR systems trained on clean speech.
• These advancements pave the way for more effective and generalizable robust ASR.