Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech...

Area of Science:

• Neuroscience
• Signal Processing
• Artificial Intelligence

Background:

• Auditory attention decoding (AAD) aims to isolate and amplify a target speaker's voice in noisy environments by analyzing listener brain activity.
• Current AAD methods rely on speech representations like waveform or spectrogram, with uncertain effectiveness.
• Self-supervised learning offers potential for more robust speech representations.

Purpose of the Study:

• To investigate the efficacy of self-supervised learned speech representations for enhancing AAD performance.
• To compare WavLM-derived speech representations against traditional methods (speech envelope, spectrogram) in AAD.

Main Methods:

• Invasive electrocorticography (ECoG) recorded brain activity from three subjects listening to two simultaneous conversations.
• WavLM was employed to extract latent speech representations.
• A spatiotemporal filter mapped ECoG data to WavLM representations for decoding.

Main Results:

• WavLM-based speech representations yielded superior decoding accuracy compared to speech envelope and spectrogram.
• AAD speed was also improved using WavLM representations.
• The study demonstrated the practical advantages of self-supervised learning in AAD.

Conclusions:

• Self-supervised learned speech representations, specifically from WavLM, offer significant advantages for auditory attention decoding.
• These findings support the development of advanced brain-controlled hearable technologies.
• The study highlights a promising direction for improving speech segregation and auditory perception.