Aligning Semantic in Brain and Language: A Curriculum Contrastive Method for Electroencephalography-to-Text...

Area of Science:

• Neuroscience
• Artificial Intelligence
• Natural Language Processing

Background:

• Electroencephalography-to-Text (EEG-to-Text) generation is crucial for Brain-Computer Interfaces (BCIs).
• A key challenge is the discrepancy between subject-specific EEG data and semantic text representations.

Purpose of the Study:

• To develop a novel strategy for recalibrating EEG representations towards semantic relevance.
• To reduce the representational gap between EEG signals and text for improved EEG-to-Text generation.

Main Methods:

• A Curriculum Semantic-aware Contrastive Learning (C-SCL) strategy was devised.
• C-SCL aligns semantically similar EEG representations and separates dissimilar ones.
• Curriculum learning was employed to create meaningful and progressive contrastive pairs.

Main Results:

• The C-SCL strategy demonstrated stable improvements across various metrics on the ZuCo benchmark.
• The method achieved new state-of-the-art results when combined with different models.
• Superior performance was observed in single-subject, low-resource, and zero-shot settings.

Conclusions:

• C-SCL effectively bridges the gap between EEG and text representations.
• The proposed method enhances EEG-to-Text generation, showing strong generalizability and effectiveness.
• This work advances the potential of BCIs through improved neural data interpretation.