Encoding voice pitch for profoundly hearing-impaired listeners

Area of Science:

• Audiology
• Speech Science
• Neuroscience

Background:

• Profound hearing impairment significantly affects speech perception.
• Auditory-visual integration is crucial for speechreading.
• Understanding prosodic elements like intonation and stress is challenging for individuals with hearing loss.

Purpose of the Study:

• To evaluate the ability of hearing-impaired subjects to track connected speech and discern intonation and stress.
• To assess the effectiveness of different auditory-visual conditions, including speechreading alone, speechreading with filtered speech, and speechreading with tonal representations of fundamental frequency (F0).
• To investigate the impact of transposed and expanded F0 ranges on speech perception.

Main Methods:

• Five profoundly hearing-impaired subjects participated.
• Conditions included speechreading alone, speechreading with low-pass filtered speech (≤4 kHz), and speechreading with matched tonal F0 signals.
• Frequency transfer functions were applied to F0, creating transposed and expanded ranges.

Main Results:

• Three subjects improved speechreading rates and prosody judgments using tonal F0 representations.
• Two subjects showed significant gains in identifying intonation and stress with expanded F0 signals alone.
• These two subjects struggled with integrating auditory and visual information for connected speech tracking, even after extensive training.

Conclusions:

• Tonal representations of F0 can significantly aid speech perception and prosody judgments in profoundly hearing-impaired individuals.
• Individual differences exist in the effectiveness of auditory-visual integration versus auditory-only F0 signal processing.
• Further research is needed to optimize auditory aids for connected speech comprehension in hearing-impaired populations.