Related Experiment Video
Updated: Oct 29, 2025
08:45
Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example
Published on: October 24, 2012
14.9K
BENDR: Using Transformers and a Contrastive Self-Supervised Learning Task to Learn From Massive Amounts of EEG Data.
Demetres Kostas1,2, Stéphane Aroca-Ouellette1,2, Frank Rudzicz1,2,3
1Department Computer Science, University of Toronto, Toronto, ON, Canada.
Frontiers in Human Neuroscience
|July 12, 2021
Summary
This study introduces a novel self-supervised approach for electroencephalography (EEG) modeling using deep neural networks, inspired by language models. The method effectively processes diverse EEG data and adapts to various brain-computer interface (BCI) tasks.
Area of Science:
- Neuroscience
- Machine Learning
- Signal Processing
Background:
- Deep neural networks (DNNs) for brain-computer interfaces (BCIs) often aim for generalizable features.
- Existing approaches may not fully leverage large, publicly available electroencephalography (EEG) datasets.
- Adapting language modeling (LM) techniques offers a promising avenue for EEG analysis.
Purpose of the Study:
- To adapt self-supervised learning methods from automatic speech recognition and LMs for EEG data.
- To develop a single pre-trained DNN model capable of processing diverse and novel EEG sequences.
- To evaluate the fine-tuning capabilities of the model for various downstream BCI and EEG classification tasks.
Main Methods:
- Utilized a self-supervised training objective, similar to LMs and automatic speech recognition.
- Adapted architectures and techniques for processing raw EEG signals.
- Pre-trained a single model on massive EEG datasets.
- Fine-tuned the model for specific BCI and EEG classification tasks.
Main Results:
- A single pre-trained model successfully processed novel EEG sequences from different hardware and subjects.
- The model's internal representations and architecture were adaptable to various downstream tasks.
- Achieved superior performance compared to task-specific self-supervision methods in sleep stage classification.
Conclusions:
- Self-supervised learning adapted from LMs provides a powerful framework for EEG analysis.
- A unified pre-trained model can generalize across diverse EEG data and tasks.
- This approach enhances the efficiency and effectiveness of BCI and EEG classification.
