MCFHNet: Multi-Channel Fusion Hybrid Network for Efficient EEG-fNIRS Multi-modal Motor Imagery Decoding

Area of Science:

• Neuroscience
• Biomedical Engineering
• Machine Learning

Background:

• Motor Imagery-based Brain Computer Interface (MI-BCI) systems decode motor intentions.
• Hybrid MI-BCI methods fuse Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) data.
• Current deep learning fusion strategies in MI-BCI are often complex.

Purpose of the Study:

• To simplify multi-modal fusion strategies for MI-BCI.
• To develop an effective deep learning model for hybrid MI decoding.
• To improve the accuracy and applicability of Brain Computer Interface systems.

Main Methods:

• Proposed a novel Multi-Channel Fusion (MCF) method to simplify data fusion.
• Designed a Multi-Channel Fusion Hybrid Network (MCFHNet) incorporating depthwise convolutions, channel attention, and Bidirectional Long Short-Term Memory (Bi-LSTM) layers.
• Evaluated MCFHNet on an open EEG-fNIRS dataset using 5-fold cross-validation for intra-subject experiments.

Main Results:

• MCFHNet demonstrated superior performance compared to existing deep learning methods.
• Achieved a high mean accuracy of 99.641% in intra-subject motor imagery decoding.
• The proposed MCF strategy simplifies fusion while maintaining high feature extraction capabilities.

Conclusions:

• The developed MCFHNet offers an effective and simplified approach for multi-modal MI decoding.
• This advancement provides a promising new option for hybrid BCI systems in rehabilitation.
• The study highlights the potential of simplified deep learning fusion for enhanced BCI performance.