Related Experiment Video
Updated: Jul 12, 2026
11:25
Simultaneous Scalp Electroencephalography (EEG), Electromyography (EMG), and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding
Published on: July 26, 2013
UMind: A unified multitask network for zero-shot M/EEG visual decoding
Chengjian Xu1, Yonghao Song2, Zelin Liao3
1Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China; Shenzhen University of Advanced Technology, China.
Summary
This study introduces UMind, a novel multitask network for decoding brain activity from EEG and MEG recordings. It enables zero-shot visual decoding tasks by learning shared neural and semantic representations.
Area of Science:
- Neuroscience
- Computer Science
- Artificial Intelligence
Background:
- Decoding brain activity from EEG and MEG is crucial for real-time brain-computer interfaces.
- Current methods often rely on single-task or task-specific models, limiting their versatility.
Purpose of the Study:
- To develop a unified multitask network (UMind) for zero-shot M/EEG visual decoding.
- To enable visual stimulus retrieval, classification, and reconstruction within a shared representation space.
Main Methods:
- UMind learns neural-visual and semantic representations via multimodal alignment with image and text data.
- Coarse and fine-grained texts enhance neural representation extraction for detailed decoding.
- A pre-trained diffusion model uses these representations for visual reconstruction.
Main Results:
- UMind demonstrates effectiveness and robustness on MEG and EEG datasets.
- The approach successfully captures spatiotemporal neural dynamics.
- It highlights the synergy of semantic information in visual feature extraction.
Conclusions:
- UMind establishes a multitask pipeline for brain visual decoding.
- The method shows biological plausibility and enhances zero-shot decoding capabilities.
- This work paves the way for more advanced brain-computer interfaces.