Modeling the effect of linguistic predictability on speech intelligibility prediction

Area of Science:

• Speech processing
• Computational linguistics
• Machine learning

Background:

• Existing speech intelligibility prediction (SIP) algorithms primarily focus on acoustic factors.
• These algorithms struggle to predict intelligibility across datasets with varying linguistic predictability.
• A need exists for SIP models that integrate linguistic information for broader applicability.

Purpose of the Study:

• To enhance existing speech intelligibility prediction (SIP) algorithms by incorporating linguistic predictability.
• To evaluate the impact of a linguistic component on SIP performance across diverse corpora.
• To improve the accuracy and robustness of speech intelligibility prediction models.

Main Methods:

• Five existing SIP algorithms were modified by integrating a linguistic predictability estimation module.
• A pre-trained language model was utilized to quantify the linguistic predictability of different corpora.
• The enhanced algorithms were tested on a mixture of four English open-set corpora with distinct linguistic characteristics.

Main Results:

• The integration of the linguistic component significantly improved the performance of the evaluated SIP algorithms.
• Enhanced correlation and reduced prediction error were observed compared to baseline acoustic-only models.
• The improved performance was consistent across the diverse English corpora used in the study.

Conclusions:

• Incorporating linguistic predictability into SIP algorithms is crucial for accurate cross-corpus prediction.
• Language models offer a viable method for estimating linguistic predictability in speech processing.
• The proposed approach advances the field of speech intelligibility prediction by addressing limitations of acoustic-only models.