Complex Spectral Mapping for Single- and Multi-Channel Speech Enhancement and Robust ASR

Area of Science:

• Signal Processing
• Machine Learning
• Acoustics

Background:

• Speech enhancement is crucial for improving the intelligibility of audio in noisy and reverberant conditions.
• Traditional methods often struggle with complex acoustic environments, limiting speech recognition accuracy.

Purpose of the Study:

• To develop an advanced speech enhancement system using deep neural networks (DNNs) for complex spectral mapping.
• To improve both speech enhancement and speech recognition performance in single- and multi-channel scenarios.

Main Methods:

• A two-stage DNN system is proposed: first, single-channel complex spectral mapping, followed by multi-channel mapping using beamforming results.
• The system predicts the real and imaginary (RI) components of the direct-path signal from noisy inputs.
• A novel time-varying beamforming method is introduced, leveraging estimated complex spectra.

Main Results:

• The proposed system achieved state-of-the-art performance on the CHiME-4 corpus for speech enhancement and recognition.
• Word error rates (WER) were significantly reduced to 6.82% (single-channel), 3.19% (two-channel), and 2.00% (six-channel), outperforming previous best results.

Conclusions:

• The complex spectral mapping approach with DNNs offers a powerful solution for robust speech enhancement.
• The method effectively utilizes spatial information and complex spectral characteristics for superior performance in challenging acoustic conditions.