Crosstalk Suppression in a Multi-Channel, Multi-Speaker System Using Acoustic Vector Sensors

Area of Science:

• Signal Processing
• Acoustics
• Machine Learning

Background:

• Automatic speech recognition (ASR) in multi-speaker, reverberant environments like courtrooms presents significant challenges.
• Crosstalk, or interference between speakers, degrades ASR performance and requires pre-processing for accurate speech-to-text transcription.

Purpose of the Study:

• To develop and evaluate an algorithm for crosstalk suppression in multi-speaker ASR scenarios.
• To enhance the performance of speech-to-text systems in challenging acoustic conditions such as conferences and court sessions.

Main Methods:

• Utilized Acoustic Vector Sensors for acquiring audio streams in multi-speaker environments.
• Employed statistical analysis of Direction of Arrival (DOA) for speaker detection and source separation.
• Implemented a dynamic gain processor for crosstalk suppression after analyzing speaker activity and identifying signal fragments.

Main Results:

• Achieved significant improvements in Scale-Invariant Signal-to-Distortion Ratio (SI-SDR).
• Demonstrated an increase of 7.54 dB in SI-SDR with the full algorithm (including source separation).
• Showcased an even higher increase of 19.53 dB in SI-SDR when only crosstalk suppression was applied.

Conclusions:

• The proposed algorithm effectively suppresses crosstalk in multi-speaker reverberant environments.
• The method enhances audio quality for subsequent speech-to-text transcription, showing promise for real-world applications.
• Source separation and dynamic gain processing are key components for achieving substantial performance gains.