NeuroDecoder: A new framework for image decoding and reconstruction of EEG signals

Area of Science:

• Neuroscience
• Computer Science
• Biomedical Engineering

Background:

• Brain-Computer Interface (BCI) technology offers significant potential for improving human health and quality of life.
• Visual stimulus reconstruction from electroencephalography (EEG) signals is a key BCI application, but is hindered by data complexity and noise.
• Existing reconstruction methods face challenges with the inherent noise and complexity of EEG data.

Purpose of the Study:

• To introduce NeuroDecoder, an end-to-end multimodal framework for high-quality image reconstruction from EEG signals.
• To address EEG noise and cross-modal discrepancies through a novel approach.
• To enhance the performance of visual stimulus reconstruction in BCI systems.

Main Methods:

• NeuroDecoder employs a three-stage process: EEG Decoding, Modality Alignment, and Image Reconstruction.
• A noise-robust encoder and a mask-based triple-contrastive learning strategy are utilized for alignment.
• A pre-trained stable diffusion model is integrated for image reconstruction without fine-tuning.

Main Results:

• NeuroDecoder achieved high subject-dependent EEG classification accuracies (up to 99.76%) and competitive subject-independent accuracies (up to 91.61%).
• The framework obtained low Fréchet Inception Distances (62.84, 63.12) for image reconstruction, indicating high quality.
• Experimental results demonstrate superior performance compared to prior methods in both EEG classification and image reconstruction.

Conclusions:

• NeuroDecoder effectively reconstructs high-quality images from EEG signals, overcoming noise and cross-modal challenges.
• The proposed framework significantly advances the capabilities of BCI technology for visual stimulus reconstruction.
• NeuroDecoder shows promise for future applications in neurofeedback, assistive technologies, and understanding brain activity.