Simultaneous EEG-fNIRS Data Classification Through Selective Channel Representation and Spectrogram Imaging.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Artificial Intelligence

Background:

• Brain-computer interfaces (BCIs) integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) shows promise.
• Existing BCIs struggle with efficient feature selection, underutilizing EEG's temporal and fNIRS's spatial data.
• This limits the performance and versatility of current EEG-fNIRS BCI systems.

Purpose of the Study:

• To develop a novel deep learning model for enhanced feature extraction and fusion in EEG-fNIRS BCIs.
• To address the limitations of current methods in leveraging the full potential of multimodal neuroimaging data.
• To improve classification accuracy and applicability across diverse cognitive and motor tasks.

Main Methods:

• Proposed the multimodal DenseNet fusion (MDNF) deep learning architecture.
• Transformed EEG data into 2D images using short-time Fourier transform.
• Integrated spectrally-enhanced EEG features with fNIRS-derived spectral entropy features using transfer learning.

Main Results:

• The MDNF model demonstrated superior performance compared to existing state-of-the-art methods on two public datasets.
• Achieved high classification accuracy by effectively utilizing both temporal and spatial features from EEG and fNIRS.
• Validated the model's effectiveness in enhancing BCI performance for various cognitive and motor imagery tasks.

Conclusions:

• The MDNF model offers a significant advancement in EEG-fNIRS BCI research by overcoming feature selection challenges.
• Its high accuracy and precise feature utilization show potential for clinical neurodiagnostics and rehabilitation.
• Paves the way for developing patient-specific therapeutic strategies using advanced BCI technology.