Deep learning based speaker separation and dereverberation can generalize across different languages to improve...

Area of Science:

• Artificial Intelligence
• Speech Processing
• Acoustics

Background:

• Deep learning models for audio processing require generalization to real-world conditions.
• Assessing model generalization across diverse acoustic environments is crucial for practical applications.

Purpose of the Study:

• To evaluate the generalization capability of a deep learning model for speaker separation and dereverberation.
• To test the model's performance across different languages, acoustic conditions, and speaker characteristics.

Main Methods:

• A deep computational auditory scene analysis algorithm was employed.
• The algorithm utilized complex time-frequency masking for magnitude and phase estimation.
• Training was conducted in English, while testing involved Mandarin, diverse speech corpora, and varied reverberation conditions.

Main Results:

• Significant improvements in speech intelligibility were observed for normal-hearing listeners across all tested conditions.
• The average intelligibility benefit was 43.5% points, comparable to same-language performance.
• The model demonstrated robust performance despite substantial differences in training and testing environments.

Conclusions:

• A well-designed deep learning network can generalize effectively across vastly different acoustic environments, including different languages.
• The findings suggest substantial potential benefits for individuals with hearing impairments.
• This research highlights the practical efficacy of deep learning for enhancing speech intelligibility in challenging acoustic scenarios.