Neural decoding of attentional selection in multi-speaker environments without access to separated sources

Area of Science:

• Neuroscience
• Biomedical Engineering
• Signal Processing

Background:

• Hearing impairments significantly impact conversation in multi-speaker environments.
• Current hearing aids struggle to isolate and amplify a specific speaker.
• Cognitively controlled hearing aids require auditory attention decoding (AAD) but face challenges with clean sound sources.

Purpose of the Study:

• To develop a novel framework combining speech separation and AAD for hearing assistance.
• To create an end-to-end system for identifying and amplifying attended speech in real-time.
• To investigate the neural basis of AAD in the auditory cortex.

Main Methods:

• Developed an end-to-end system integrating single-channel speech separation with AAD.
• Utilized invasive electrophysiology recordings to capture neural signals.
• Employed the system to decode auditory attention from mixed audio signals.

Main Results:

• Successfully decoded subject attention and detected attention switches using only mixed audio.
• Identified specific auditory cortex regions involved in AAD.
• Demonstrated significant improvements in subjective and objective speech quality measures.

Conclusions:

• The proposed framework effectively combines speech separation and AAD for hearing aid applications.
• This approach bridges advancements in speech processing and hearing prosthesis research.
• The developed system moves closer to realizing cognitively controlled hearing aids.