EEG-based auditory attention decoding with audiovisual speech for hearing-impaired listeners

Area of Science:

• Neuroscience
• Auditory Neuroscience
• Speech Perception

Background:

• Auditory attention decoding (AAD) identifies attended speech but its modulation in hearing-impaired (HI) listeners is unclear.
• Understanding factors influencing AAD is crucial for developing assistive listening technologies for HI individuals.

Purpose of the Study:

• To investigate how auditory and visual factors modulate AAD in hearing-impaired listeners.
• To correlate AAD performance with individual auditory abilities.

Main Methods:

• Scalp electroencephalogram (EEG) recorded during an auditory selective attention task with varying visual input and target-to-masker ratios (TMR).
• Behavioral hearing tests including audiogram, speech reception threshold, and temporal modulation transfer function were administered.
• Temporal response function analysis examined cortical tracking and AAD accuracy.

Main Results:

• Both visual input and increased TMR significantly improved cortical tracking of attended speech and AAD accuracy.
• Audiovisual (AV) gain correlated with amplitude modulation (AM) sensitivity; TMR gain correlated with hearing thresholds.
• Higher AM sensitivity was linked to greater AV gain in specific brain regions.

Conclusions:

• Visual cues and TMR are significant modulators of AAD in hearing-impaired listeners.
• Individual auditory processing abilities, particularly AM sensitivity, play a key role in audiovisual speech perception and AAD.
• Findings inform strategies for enhancing speech understanding in complex auditory environments for HI individuals.