Exploring the Potential of Electroencephalography Signal-Based Image Generation Using Diffusion Models: Integrative...

Area of Science:

• Neuroscience
• Computer Vision
• Machine Learning

Background:

• Electroencephalography (EEG) measures brain activity but faces challenges in accurate image generation from neural signals.
• Existing research primarily focuses on EEG signal classification, with limited exploration of EEG-based image generation.
• Generating meaningful images directly from EEG remains an underexplored area.

Purpose of the Study:

• To advance EEG-based classification towards direct image generation.
• To develop a novel framework, NECOMIMI, for synthesizing images from EEG signals.
• To overcome limitations of previous methods in EEG-to-image translation.

Main Methods:

• A 2-stage NECOMIMI method integrating a custom Neural Encoding Representation Vectorizer (NERV) EEG encoder with a diffusion-based generative model.
• Introduction of the Category-Based Assessment Table (CAT) score for evaluating semantic quality of EEG-generated images.
• Validation and benchmarking of the CAT score using the ThingsEEG dataset.

Main Results:

• The NERV EEG encoder achieved state-of-the-art performance in zero-shot classification tasks (e.g., 94.8% accuracy in 2-way task).
• The 2-stage NECOMIMI architecture effectively extracted semantic information from noisy EEG signals, generating more specific images compared to 1-stage methods.
• Perturbation studies indicated reliance on posterior brain signals for semantically coherent image generation.

Conclusions:

• NECOMIMI demonstrates the potential and challenges of EEG-to-image generation.
• The NERV encoder achieved state-of-the-art results in zero-shot classification and EEG-informed image generation.
• The CAT score offers a new evaluation metric, and the technology holds significant clinical potential for brain-machine interfaces.