Fusing Bone-conduction and Air-conduction Sensors for Complex-Domain Speech Enhancement

Area of Science:

• Signal Processing
• Acoustics
• Machine Learning

Background:

• Speech enhancement is crucial for improving intelligibility in noisy conditions.
• Low signal-to-noise ratio (SNR) presents significant challenges for conventional methods.
• Air-conduction (AC) microphones capture full-band speech but are noise-sensitive; bone-conduction (BC) sensors are noise-immune but have limited bandwidth.

Purpose of the Study:

• To develop an effective speech enhancement method by fusing AC and BC signals.
• To address the challenge of limited bone-conduction data.
• To improve speech intelligibility in adverse acoustic environments.

Main Methods:

• An attention-based fusion model was proposed to combine AC and BC signals for complex spectral mapping.
• Experiments were conducted on the EMSB dataset.
• A semi-supervised technique was developed to leverage both parallel and unparallel AC and BC speech data, incorporating AISHELL-1 data.

Main Results:

• The proposed attention-based fusion method outperformed a time-domain baseline across all tested conditions.
• Sensor fusion demonstrated superiority over single-sensor approaches, particularly in low SNR scenarios.
• The semi-supervised technique achieved performance comparable to supervised learning using only 50% of the parallel data.

Conclusions:

• Attention-based fusion of AC and BC signals is an effective strategy for robust speech enhancement.
• The proposed method significantly improves speech quality and intelligibility in challenging, noisy environments.
• Semi-supervised learning offers a viable solution for utilizing limited BC data in speech enhancement systems.