Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in...

Area of Science:

• Neuroscience
• Biomedical Engineering
• Signal Processing

Background:

• Brain-computer interfaces (BCIs) show promise for emotion recognition, particularly using functional near-infrared spectroscopy (fNIRS).
• Current fNIRS-based BCIs are limited by insufficient feature extraction algorithms.
• Integrating electroencephalography (EEG) signals offers a potential avenue for improvement.

Purpose of the Study:

• To enhance the performance of fNIRS-based BCIs for emotion recognition.
• To introduce a novel feature extraction method combining fNIRS and EEG signals.
• To investigate the application of transfer learning between EEG and fNIRS data.

Main Methods:

• Proposed the R-CSP-E method, integrating EEG signals into fNIRS feature computation using transfer and ensemble learning.
• Employed Independent Component Analysis (ICA) for signal source correspondence.
• Utilized a modified Common Spatial Pattern (CSP) algorithm incorporating EEG signals for spatial filtering.

Main Results:

• The R-CSP-E method demonstrated superior performance compared to traditional approaches without transfer learning.
• Mean classification accuracy was increased by up to 5% on public datasets.
• Successfully applied transfer learning between EEG and fNIRS signals, a novel approach.

Conclusions:

• The R-CSP-E method significantly improves fNIRS-based BCI performance for emotion recognition.
• Transfer learning is a viable and effective strategy for cross-modal BCIs.
• This research offers a new perspective for hybrid BCI development.