Speech2EEG: Leveraging Pretrained Speech Model for EEG Signal Recognition

Area of Science:

• Neuroscience
• Machine Learning
• Signal Processing

Background:

• Brain-computer interface (BCI) applications require accurate recognition of electroencephalography (EEG) signals.
• Current neural network approaches for EEG recognition often necessitate complex structures and sufficient training data, posing a challenge due to data scarcity.

Purpose of the Study:

• To introduce Speech2EEG, a novel method for EEG recognition that utilizes pretrained speech features to improve accuracy, particularly in data-limited scenarios.
• To explore effective strategies for integrating multichannel temporal embeddings extracted from EEG signals.

Main Methods:

• Adapted a pretrained speech processing model to extract multichannel temporal embeddings from EEG signals.
• Implemented various aggregation methods (weighted average, channelwise, channel-and-depthwise) to integrate these embeddings.
• Utilized a classification network to predict EEG categories based on the integrated features.

Main Results:

• Speech2EEG achieved state-of-the-art performance on the BCI IV-2a and BCI IV-2b motor imagery datasets, with accuracies of 89.5% and 84.07%, respectively.
• Visualization confirmed that the Speech2EEG architecture effectively captures relevant patterns for motor imagery classification.
• The method demonstrates efficacy even with limited dataset scales.

Conclusions:

• Speech2EEG offers a novel and effective solution for EEG signal analysis, particularly for motor imagery tasks within brain-computer interfaces.
• Leveraging pretrained speech models presents a promising avenue for improving EEG recognition accuracy and addressing data limitations.
• The proposed integration techniques for multichannel temporal embeddings provide valuable insights for future research in the field.