ATTENTION-BASED FUSION FOR BONE-CONDUCTED AND AIR-CONDUCTED SPEECH ENHANCEMENT IN THE COMPLEX DOMAIN.

Area of Science:

• Signal Processing
• Audio Engineering
• Machine Learning for Audio

Background:

• Bone-conduction (BC) microphones capture skull vibrations for speech, offering noise immunity but limited bandwidth.
• Air-conduction (AC) microphones capture full-band speech but are highly sensitive to acoustic noise.
• Existing methods struggle to effectively combine BC and AC signals for optimal speech enhancement.

Purpose of the Study:

• To develop an advanced speech enhancement system integrating both BC and AC microphone signals.
• To leverage the complementary strengths of BC and AC microphones for improved speech intelligibility.
• To outperform existing single-microphone and hybrid speech enhancement techniques.

Main Methods:

• Utilized a convolutional recurrent network for complex spectral mapping of audio signals.
• Implemented attention-based fusion strategies, including early-fusion and late-fusion, to integrate BC and AC data.
• Trained and evaluated the model on speech signals under various noise conditions.

Main Results:

• The proposed attention-based fusion method demonstrated superior performance compared to other BC-AC integration techniques.
• Significant improvements in speech enhancement were observed, particularly in low signal-to-noise ratio (SNR) environments.
• The method outperformed conventional speech enhancement approaches in challenging acoustic scenarios.

Conclusions:

• Combining BC and AC microphones with a convolutional recurrent network and attention-based fusion is highly effective for speech enhancement.
• The proposed approach offers robust performance, especially in noisy conditions where traditional methods falter.
• This hybrid microphone strategy represents a significant advancement in overcoming limitations of individual microphone types for clear speech.