Related Experiment Video
Updated: Jun 4, 2025
Author Spotlight: Enhancing Neurorehabilitation Through EEG, Motor Imagery, and Virtual Reality
Published on: May 10, 2024
A bimodal deep learning network based on CNN for fine motor imagery.
Chenyao Wu1,2, Yu Wang2,3, Shuang Qiu1,2
1Laboratory of Brain Atlas and Brain-Inspired Intelligence, Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190 China.
This study introduces a novel bimodal fusion network combining electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) to enhance brain-computer interface (BCI) control using fine motor imagery (MI). The method significantly improves decoding accuracy for complex movements.
Area of Science:
- Neuroscience
- Biomedical Engineering
- Signal Processing
Background:
- Motor imagery (MI) is crucial for brain-computer interfaces (BCIs), but traditional paradigms have limitations.
- Fine MI, focusing on joint movements, offers more intuitive control but faces decoding challenges.
- Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are portable sensing modalities for BCI systems.
Purpose of the Study:
- To design and evaluate a fine MI paradigm for controlling mechanical devices.
- To investigate the feasibility of combining EEG and fNIRS for enhanced BCI performance.
- To develop and validate a bimodal fusion network for improved fine MI decoding.
Main Methods:
- A four-class fine MI paradigm (hand, wrist, shoulder, rest) was implemented.
- EEG and fNIRS data were collected from 12 subjects during MI tasks.
- A bimodal fusion network utilizing convolutional neural networks (CNNs) was proposed for feature extraction and classification.
Main Results:
- Distinct EEG event-related desynchronization (ERD) and fNIRS activation patterns were observed for the four MI classes.
- The proposed bimodal fusion network significantly outperformed single-modal approaches.
- A four-class accuracy of 58.96% was achieved, demonstrating superior decoding performance.
Conclusions:
- EEG and fNIRS bimodal BCI systems are feasible for fine MI tasks.
- The proposed bimodal fusion method effectively enhances decoding accuracy.
- This research provides valuable techniques for advancing fine MI-based BCI systems.
