Recognizing Natural Images From EEG With Language-Guided Contrastive Learning.

Area of Science:

• Neuroscience
• Computer Science
• Artificial Intelligence

Background:

• Electroencephalography (EEG) offers noninvasive brain signal acquisition but faces challenges in signal quality and decoding accuracy for complex tasks like image recognition.
• Previous research has shown limited success in decoding image information from EEG, primarily due to performance limitations and questions of biological plausibility.

Purpose of the Study:

• To introduce and validate a self-supervised learning framework for recognizing images from EEG signals.
• To enhance the semantic understanding of EEG data by integrating large language models (LLMs).
• To provide evidence for the feasibility and biological plausibility of EEG-based image recognition.

Main Methods:

• A self-supervised framework leveraging contrastive learning to align EEG responses with image stimuli.
• Integration of language descriptions generated by LLMs to guide the learning of core semantic information.
• Validation on the THINGS-EEG2 dataset using zero-shot learning tasks.

Main Results:

• Achieved significantly above-chance performance on the THINGS-EEG2 dataset, with top-1 accuracy of 19.7% and top-5 accuracy of 51.5% in 200-way zero-shot tasks.
• Experimental analysis from temporal, spatial, spectral, and semantic perspectives supported the biological plausibility of the findings.
• Comparative studies with the THINGS-Magnetoencephalography (MEG) dataset corroborated the results.

Conclusions:

• The developed framework demonstrates the feasibility of recognizing images from EEG signals, enhanced by LLM-guided semantic learning.
• The results provide strong evidence for the biological plausibility of EEG-based image recognition.
• The findings have significant implications for advancing neural decoding and brain-computer interfaces (BCIs) in areas like healthcare and robotics.