A two-stage EEG zero-shot classification algorithm guided by class reconstruction

Objective. Researchers have long been dedicated to decoding human visual representations from neural signals. These studies are crucial in uncovering the mechanisms of visual processing in the human brain. Electroencephalogram (EEG) signals have garnered widespread attention recently due to their non-invasive nature and low cost. EEG classification is one of the most popular topics in brain-computer interface research. However, most traditional EEG classification algorithms are difficult to generalize to unseen classes that were not involved in the training phase. The main objective of this work is to improve the performance of these EEG classification algorithms for unseen classes.Approach. In this work, we propose a two-stage zero-shot EEG classification algorithm guided by class reconstruction. The method is specifically designed with a two-stage training strategy based on class reconstruction. This structure and training strategy enable the model to thoroughly learn the relations and distinctions among EEG embeddings of different classes. The contrastive language-image pre-training (CLIP) model has a well-aligned latent space and powerful cross-modality generalization ability.Main results. We conducted experiments on the ImageStimulus-EEG dataset to evaluate the performance of the proposed method. Meanwhile, it was compared with the state-of-the-art model and the baseline model. The experimental results demonstrate that our model achieves superior performance in among Top-1, Top-3, and Top-5 classification accuracy for a 50-way zero-shot classification task, reaching 17.77%, 38.76% and 54.75%, respectively.Significance. The proposed method bridges the modality gap between EEG, images, and text using CLIP features. It significantly improves the model's performance in unseen classes. The experimental results validate the effectiveness of it in EEG zero-shot classification.