Neural decoding of attentional selection in multi-speaker environments without access to clean sources

Area of Science:

• Neuroscience
• Auditory Neuroscience
• Signal Processing

Background:

• Hearing impairments significantly impact conversational ability in multi-speaker settings.
• Existing hearing aids struggle to isolate and amplify specific conversations.
• Cognitively controlled hearing aids utilizing auditory attention decoding (AAD) offer a promising advancement.

Purpose of the Study:

• To develop a novel framework integrating speech separation and AAD for hearing assistance.
• To address the challenge of accessing clean sound sources in real-world AAD applications.

Main Methods:

• An end-to-end system was developed to process mixed audio and neural signals.
• The system automatically separates individual speakers from a single audio channel.
• Invasive electrophysiology recordings identified auditory cortex regions crucial for AAD.

Main Results:

• The system successfully decodes user attention and amplifies the attended speaker's voice.
• Significant improvements in subjective and objective speech quality measures were achieved.
• The framework operates effectively using only the mixed audio input.

Conclusions:

• The proposed AAD framework bridges speech processing and hearing prosthesis research.
• This work advances the development of cognitively controlled hearable devices.
• It offers a pathway to improved hearing assistance for individuals with hearing impairments.