Syllable-rate-adjusted-modulation (SRAM) predicts clear and conversational speech intelligibility

Area of Science:

• Speech processing
• Acoustic phonetics
• Signal processing

Background:

• Objective prediction of speech intelligibility is crucial for telecommunication and human-machine interaction.
• Traditional methods using signal-to-noise ratios (SNR) struggle to account for the enhanced intelligibility of clear speech.
• No existing objective metric reliably predicts the clear speech benefit at the sentence level.

Purpose of the Study:

• To propose and validate a novel objective metric, the Syllable-Rate-Adjusted-Modulation (SRAM) index, for predicting speech intelligibility.
• To evaluate SRAM's performance against established intelligibility metrics and state-of-the-art automatic speech recognition systems.
• To demonstrate the significance of syllable rate in intelligibility prediction.

Main Methods:

• Developed the Syllable-Rate-Adjusted-Modulation (SRAM) index, which estimates modulation power above the syllable rate from short speech segments (as brief as 1 second).
• Compared SRAM against three reference metrics: envelope-regression-based speech transmission index (ER-STI), hearing-aid speech perception index version 2 (HASPI-v2), and short-time objective intelligibility (STOI).
• Evaluated SRAM against five automatic speech recognition systems: Amazon Transcribe, Microsoft Azure Speech-To-Text, Google Speech-To-Text, wav2vec2, and Whisper.

Main Results:

• The proposed SRAM index significantly outperformed ER-STI, HASPI-v2, and STOI in predicting speech intelligibility.
• SRAM demonstrated superior performance compared to all five evaluated automatic speech recognition systems.
• Comparison with total modulation power (TMP) confirmed the critical role of syllable rate adjustment in SRAM's effectiveness.

Conclusions:

• The SRAM index offers a promising objective measure for predicting the intelligibility of both clear and conversational speech.
• SRAM's ability to capture the clear speech benefit has implications for understanding speech production and perception.
• Potential applications include screening speech materials for high intelligibility and developing methods to convert conversational speech to clear speech.