A Novel Speech Intelligibility Enhancement Model based on Canonical Correlation and Deep Learning

Area of Science:

• Speech processing
• Machine learning
• Acoustics

Background:

• Current deep learning (DL) models for speech enhancement often minimize feature distance, improving quality but not intelligibility in real-world noise.
• Intelligibility-oriented (I-O) loss functions offer a promising direction for robust speech enhancement.

Purpose of the Study:

• To develop and evaluate a novel canonical correlation-based intelligibility-oriented loss function for deep learning-based speech enhancement.
• To improve speech intelligibility in noisy environments beyond conventional methods.

Main Methods:

• Formulation of a canonical correlation-based short-time objective intelligibility (CC-STOI) cost function.
• Training a fully convolutional neural network (FCN) model using the proposed CC-STOI loss.
• Comparative simulation experiments against state-of-the-art DL models using distance-based and STOI-based losses.

Main Results:

• The CC-STOI based speech enhancement framework significantly outperformed conventional DL models.
• Superior performance was observed in both objective and subjective evaluations.
• Effectiveness was demonstrated for unseen speakers and diverse noise conditions.

Conclusions:

• The proposed CC-STOI loss function offers a more effective approach for training DL models for robust speech enhancement.
• This method shows potential for enhancing speech intelligibility in challenging acoustic environments.
• Future work includes applying this approach to hearing-assistive technology.