Toward Domain-Free Transformer for Generalized EEG Pre-Training

Area of Science:

• Neuroscience
• Machine Learning
• Signal Processing

Background:

• Electroencephalography (EEG) signals are vital for monitoring human physiological states due to their non-invasive nature and portability.
• Deep learning models show promise for decoding complex EEG information, but require large datasets for training.
• Existing pre-trained EEG models face limitations, including dataset configuration constraints and the need for data distortion.

Purpose of the Study:

• To introduce DFformer, a novel domain-free transformer designed to generalize pre-trained electroencephalography (EEG) models.
• To develop a pre-trained model using DFformer that integrates seamlessly across diverse EEG datasets without architectural changes or data manipulation.
• To address the challenge of data scarcity in training deep learning models for EEG signal analysis.

Main Methods:

• Proposed a domain-free transformer architecture named DFformer.
• Developed a pre-trained model based on DFformer for EEG signal analysis.
• Evaluated the model's performance on motor imagery and sleep stage classification tasks using diverse datasets.

Main Results:

• The DFformer-based pre-trained model demonstrated seamless integration across different EEG datasets.
• Achieved competitive performance in motor imagery and sleep stage classification.
• Showed significant performance improvements when fine-tuned on datasets distinct from the pre-training set.

Conclusions:

• DFformer effectively overcomes limitations of existing pre-trained EEG models, enabling generalization across diverse datasets.
• The proposed model offers robust applicability in various domains requiring EEG analysis.
• DFformer facilitates improved performance in EEG-based deep learning tasks, especially with limited data.