Auditory Attention Detection via Cross-Modal Attention

Area of Science:

• Neuroscience
• Signal Processing
• Machine Learning

Background:

• Humans possess advanced auditory attention capabilities to isolate desired speech streams in noisy environments.
• Auditory attention detection (AAD) using electroencephalography (EEG) shows promise but struggles with short signal segments and real-time applications.
• Existing AAD methods lack robustness, necessitating advanced decoding strategies for reliable performance.

Purpose of the Study:

• To develop a novel cross-modal attention-based AAD (CMAA) approach.
• To leverage correlations between audio and EEG signals for enhanced attention detection.
• To improve the accuracy and robustness of real-time AAD.

Main Methods:

• Proposed a cross-modal attention-based AAD (CMAA) network.
• Exploited discriminative features and cross-modal correlations between audio and EEG.
• Utilized data visualization and comprehensive experiments on a public database.

Main Results:

• Achieved high accuracy rates: 82.8% (1s), 86.4% (2s), and 87.6% (5s) under anechoic conditions.
• Attained 84.1% accuracy with a 2s window in reverberant conditions.
• Outperformed conventional linear and state-of-the-art non-linear AAD approaches.

Conclusions:

• The CMAA model effectively fuses cross-modal information by attending to audio and EEG features.
• CMAA demonstrates superior performance in auditory attention detection compared to existing methods.
• The findings support the potential of CMAA for robust real-time AAD applications.