A lightweight speech enhancement network fusing bone- and air-conducted speech

Area of Science:

• Signal Processing
• Acoustics
• Machine Learning

Background:

• Air-conducted (AC) microphones capture clear speech but also ambient noise.
• Bone-conducted (BC) microphones are noise-immune but offer limited speech bandwidth.
• Integrating AC and BC speech offers a potential solution for robust speech enhancement.

Purpose of the Study:

• To develop a speech enhancement model that synergistically utilizes both AC and BC speech signals.
• To improve the quality of speech in noisy environments by mitigating ambient noise interference.
• To leverage attention mechanisms for effective feature fusion and speech signal reconstruction.

Main Methods:

• Inputting spectrograms of BC and AC speech into an attention-based feature fusion module.
• Employing a densely gated convolutional attention network (DenGCAN) as the backbone for speech mask estimation.
• Integrating an improved attention gate into DenGCAN's skip-connections to enhance feature focus.

Main Results:

• The proposed model successfully fuses AC and BC speech features.
• The DenGCAN backbone with enhanced attention gates effectively estimates speech masks.
• Experimental results show an average wideband-PESQ improvement of 1.870 over noisy AC speech.

Conclusions:

• The proposed speech enhancement model effectively leverages the complementary strengths of AC and BC speech.
• The DenGCAN architecture with attention gates provides a lightweight yet powerful solution for noise reduction.
• This approach offers significant improvements in speech enhancement, particularly in noisy conditions.