DAMind: Zero-Shot Visual Cross-Domain Alignment and Representation for EEG Decoding
Summary
This study introduces DAMind, a novel multimodal electroencephalography (EEG) model for decoding visual cognition. DAMind effectively aligns and decodes brain signals across different domains, improving performance on unseen visual tasks.
Area of Science:
- Neuroscience
- Artificial Intelligence
- Computer Vision
Background:
- Decoding visual cognition from brain signals is vital for human assistance.
- Current brain decoding models struggle with small datasets and lack cross-domain generalization.
- Existing methods fail to learn uniform representations across different data domains, degrading performance.
Purpose of the Study:
- To propose DAMind, a multimodal EEG-based model for robust visual cross-domain alignment and decoding.
- To leverage Vision-Language Models (VLMs) and brain-inspired mechanisms for enhanced feature extraction.
- To achieve effective cross-domain zero-shot transfer for brain decoding.
Main Methods:
- Integrating VLMs with brain-inspired cognitive mechanisms for feature extraction.
- Utilizing a visual guidance mechanism for effective visual fine-tuning.
- Implementing a stepwise EEG encoding process aligned with visual processing and instruction-based learning.
Main Results:
- DAMind demonstrates robust architecture for mapping EEG signals from multiple domains to a unified learning domain.
- Achieved state-of-the-art results on several visual tasks within the comprehensive EEG decoding benchmark, EBench.
- Outperformed baseline models in zero-shot settings, showcasing strong generalization capabilities.
Conclusions:
- DAMind offers a robust solution for cross-domain visual decoding using EEG signals.
- The model effectively learns both low-level visual features and high-level semantic concepts from neural data.
- DAMind advances the field of brain-computer interfaces by enabling effective zero-shot transfer in visual tasks.
