EEG-CLIP: A transformer-based framework for EEG-guided image generation.

Area of Science:

• Neuroscience and Artificial Intelligence
• Brain-Computer Interfaces (BCIs)

Background:

• Functional magnetic resonance imaging (fMRI) shows promise for neural decoding but faces practical deployment and cost challenges.
• Electroencephalography (EEG) offers superior temporal resolution, portability, and cost-effectiveness for real-time BCIs.
• Existing EEG-based methods are limited by inadequate architectures, low reconstruction fidelity, and inconsistent evaluation.

Purpose of the Study:

• To introduce EEG-CLIP, a novel Transformer-based framework to enhance visual perception decoding from EEG signals.
• To address limitations in architectural design, reconstruction fidelity, and evaluation protocols of current EEG-BCI systems.
• To achieve state-of-the-art performance in both classification and image reconstruction tasks using EEG data.

Main Methods:

• Developed a specialized EEG-ViT encoder for capturing spatial-temporal EEG characteristics and a Diffusion Prior Transformer.
• Implemented a dual-stage reconstruction pipeline using class contrastive learning and pretrained diffusion models.
• Established comprehensive evaluation protocols across multiple datasets, including temporal window sensitivity and brain activation visualization.

Main Results:

• EEG-CLIP demonstrates robust performance, with improvements attributed to its specialized architecture and training techniques.
• Achieved state-of-the-art quantitative and qualitative results on ThingsEEG and Brain2Image datasets.
• Successfully advanced neural signal-based visual decoding capabilities.

Conclusions:

• EEG-CLIP represents a significant advancement in EEG-based visual decoding for brain-computer interfaces.
• The framework overcomes previous limitations, offering enhanced reconstruction fidelity and robust performance.
• This work paves the way for more sophisticated and practical EEG-BCI applications.