Emotion Recognition Based on a EEG-fNIRS Hybrid Brain Network in the Source Space

Area of Science:

• Neuroscience
• Biomedical Engineering
• Signal Processing

Background:

• Multimodal physiological signals like EEG and fNIRS enhance emotion recognition over unimodal methods.
• Limited research exists on the relationship between EEG and fNIRS for brain network construction in emotion recognition.

Purpose of the Study:

• To develop a novel method for constructing hybrid brain networks using simultaneous EEG-fNIRS signals.
• To improve emotion recognition performance by integrating causal and coupled brain networks.

Main Methods:

• Source localization of EEG signals to derive source signals.
• Granger causality analysis for EEG source signals to establish causal brain networks.
• Coupling strength assessment between EEG and fNIRS signals for coupled brain networks.
• Integration of causal and coupled networks into hybrid brain networks for feature extraction.

Main Results:

• The proposed hybrid brain network method was validated on multiple emotion datasets.
• Experimental results demonstrated significantly superior emotion recognition performance compared to baseline methods.

Conclusions:

• This work presents a novel perspective on fusing EEG and fNIRS signals for emotion recognition.
• The study provides a feasible solution for enhancing emotion recognition accuracy through hybrid brain network construction.