Measuring time-frequency importance functions of speech with bubble noise

Area of Science:

• Auditory perception
• Speech processing
• Signal processing

Background:

• Listeners can understand speech in noisy environments, but the specific acoustic features utilized remain unclear.
• Identifying these features is crucial for developing advanced speech enhancement technologies.

Purpose of the Study:

• To introduce a data-driven framework for identifying time-frequency regions critical for speech intelligibility in noise.
• To investigate the role of spectro-temporal modulation alignment between speech and noise in determining intelligibility.

Main Methods:

• A framework was developed to compute the intelligibility contribution of each time-frequency point in speech mixtures.
• Speech utterances were mixed with diverse noise instances at consistent global signal-to-noise ratios.
• Classification models were trained to predict intelligibility based on identified speech feature locations ('glimpses').

Main Results:

• The framework successfully identified time-frequency locations of noise-robust phonetic features within syllables.
• The alignment between speech and spectro-temporally modulated noise significantly impacts speech intelligibility.
• Trained classification models demonstrated generalization to novel noise, talkers, and speech conditions.

Conclusions:

• The study reveals specific phonetic features crucial for speech perception in noise.
• The developed framework accurately predicts speech intelligibility and generalizes to new scenarios.
• Findings advance our understanding of auditory processing and inform the design of hearing aids and speech recognition systems.