Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition.

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• Joint training of speech enhancement (SE) and automatic speech recognition (ASR) models is crucial for improving speech processing performance.
• Leveraging linguistic information from ASR to SE models can enhance SE capabilities.
• Existing methods may not fully capture linguistic nuances or can overfit to noisy data.

Purpose of the Study:

• To propose a novel joint training approach for SE and ASR models.
• To introduce an acoustic tokenizer and a cluster-based pairwise contrastive (CBPC) loss function for effective linguistic information transfer.
• To improve speech enhancement quality and reduce word error rate (WER) in noisy environments.

Main Methods:

• A pipeline integrating SE and ASR models with an acoustic tokenizer was developed.
• The acoustic tokenizer generated pseudo-labels using K-means clustering from ASR encoder outputs.
• A novel cluster-based pairwise contrastive (CBPC) loss function, combined with infoNCE, was proposed for self-supervised learning to transfer linguistic information.

Main Results:

• The proposed joint training approach with CBPC loss achieved a lower word error rate (WER) compared to conventional methods.
• Speech quality scores were significantly higher than standalone SE models and those trained with traditional joint approaches.
• The combination of CBPC and infoNCE loss demonstrated effectiveness in reducing WER and enhancing speech quality.

Conclusions:

• The proposed CBPC loss function effectively transfers linguistic information from ASR to SE models within a joint training framework.
• This approach leads to superior speech enhancement performance and more accurate speech recognition in noisy conditions.
• The method offers a promising direction for advancing robust speech processing systems.