BELT: Bootstrapped EEG-to-Language Training by Natural Language Supervision

Area of Science:

• Neuroscience
• Artificial Intelligence
• Signal Processing

Background:

• Decoding natural language from brain signals via brain-computer interfaces (BCIs) is a growing field.
• Current electroencephalography (EEG) signal decoding methods have limitations in sentence-level accuracy.
• Learning generalizable EEG representations is challenging due to small dataset sizes.

Purpose of the Study:

• To enhance EEG encoders for improved natural language decoding performance.
• To introduce a novel discrete Conformer encoder (D-Conformer) for EEG signal processing.
• To mitigate challenges associated with small EEG datasets and improve semantic representation learning.

Main Methods:

• Developed a discrete Conformer encoder (D-Conformer) to transform EEG signals into discrete representations.
• Implemented an early-stage EEG-language alignment strategy to bootstrap the learning process.
• Evaluated the D-Conformer's effectiveness through extensive experiments and ablation studies.

Main Results:

• The D-Conformer captures local and global EEG patterns, creating resilient discrete representations.
• Early-stage alignment addresses small dataset limitations, facilitating semantic representation learning.
• Demonstrated superior performance in word-level, sentence-level (5.45% BLEU-1 improvement), and sentiment-level (14% accuracy improvement) decoding tasks.

Conclusions:

• The proposed D-Conformer encoder and bootstrapping strategy significantly enhance EEG-based natural language decoding.
• The method yields more distinctive EEG representations, outperforming existing EEG encoders.
• This work advances BCI capabilities for more effective communication through brain signals.