Fine-grained image generation with EEG multi-level semantics

Area of Science:

• Neuroscience
• Artificial Intelligence
• Computer Vision

Background:

• Decoding visual information from electroencephalography (EEG) signals is critical for advancing neuroscience and artificial intelligence.
• Current methods struggle to extract fine-grained visual attributes like color distribution from EEG data, limiting image generation capabilities.

Purpose of the Study:

• To introduce EEG2IM, a novel framework for fine-grained image generation guided by multi-level EEG semantic features.
• To enhance the extraction and integration of both high-level and low-level EEG features for precise image synthesis.

Main Methods:

• EEG2IM employs a high-level semantic encoder trained via knowledge distillation and a low-level semantic encoder for fine-grained attribute extraction.
• Multi-level EEG features are integrated into a diffusion model using Feature-wise Linear Modulation (FiLM) for controlled image synthesis.
• The framework aligns EEG features with image features using an autoencoder via joint training.

Main Results:

• EEG2IM achieved high accuracy in classification tasks, reaching 99.95% on ImageNet-40 and 92.55% on ImageNet-4.
• For image generation, EEG2IM outperformed existing methods with an Inception Score (IS) of 17.58 and Fréchet Inception Distance (FID) of 52.84 on ImageNet-40.
• On ImageNet-4, EEG2IM achieved an IS of 8.79 and FID of 19.49, demonstrating superior generation quality.

Conclusions:

• EEG2IM effectively captures both high-level semantics and low-level details from EEG signals.
• The framework represents a significant advancement in fine-grained EEG-based image generation.
• The results underscore the potential of integrating multi-level EEG features for sophisticated AI applications.