Related Experiment Video
Updated: Feb 28, 2026
11:28
Concurrent EEG and Functional MRI Recording and Integration Analysis for Dynamic Cortical Activity Imaging
Published on: June 30, 2018
12.4K
Decoupled Bidirectional Spatio-Temporal Fusion Network for Hybrid EEG-fNIRS Cognitive Task Classification
Zirui Wang1, Guanghao Huang1, Zhuochao Chen1
1Institute for Future, School of Automation, Qingdao University, Qingdao 266071, China.
Brain Sciences
|February 27, 2026
Summary
This study introduces BiSTF-Net, a novel method for fusing electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals. The new approach significantly improves cognitive task recognition accuracy using multimodal neuroimaging.
Area of Science:
- Neuroimaging
- Cognitive Neuroscience
- Biomedical Engineering
Background:
- Multimodal neuroimaging, integrating electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), is crucial for brain function studies.
- Significant spatio-temporal heterogeneity between EEG and fNIRS signals presents challenges for efficient data fusion.
- Cognitive task recognition requires robust methods for combining diverse neural data streams.
Purpose of the Study:
- To present the BiSTF-Net, a novel deep learning architecture for enhanced cognitive task recognition.
- To address the challenge of fusing heterogeneous EEG and fNIRS signals for improved classification accuracy.
- To develop a robust and interpretable solution for multimodal neuroimaging data analysis.
Main Methods:
- Implemented a BiSTF-Net architecture featuring decoupled, bi-directional spatio-temporal fusion.
- Utilized bi-directional cross-modal guidance (Bi-CMG) for mutual enhancement of spatial features between EEG and fNIRS.
- Employed adaptive temporal alignment (ATA) for data-driven alignment of fNIRS signal latencies and symmetric cross-attention fusion (SCAF) for deep feature fusion.
Main Results:
- BiSTF-Net achieved high average accuracies: 83.33% for mental arithmetic (MA), 82.09% for motor imagery (MI), and 84.99% for word generation (WG).
- The proposed method demonstrated superior performance compared to existing techniques in cognitive task classification.
- The fusion strategy resulted in a modality-invariant and discriminative representation of neural activity.
Conclusions:
- BiSTF-Net offers a superior, robust, and interpretable approach for multimodal EEG-fNIRS cognitive task classification.
- The method provides a strong foundation for future research in multimodal data fusion and clinical applications.
- The spatio-temporal fusion mechanisms effectively address the heterogeneity of EEG and fNIRS signals.
