Temporal attention fusion network with custom loss function for EEG-fNIRS classification.

Area of Science:

• Neuroscience and Biomedical Engineering
• Brain-Computer Interfaces
• Neurological Disorder Diagnostics

Background:

• Accurate detection of brain activity is vital for understanding and managing neurological disorders.
• Combining electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) offers synergistic advantages over individual modalities.
• Existing multimodal EEG-fNIRS approaches face challenges with class imbalance and inter-subject variability.

Purpose of the Study:

• To develop a novel Temporal Attention Fusion Network (TAFN) for enhanced multimodal brain activity analysis.
• To address class imbalance and inter-class variability in EEG-fNIRS data using a custom loss function.
• To improve the accuracy of detecting cognitive and motor intentions and subtle neurological patterns.

Main Methods:

• Proposed a Temporal Attention Fusion Network (TAFN) integrating attention mechanisms with LSTM and temporal convolutional layers.
• Developed a custom loss function incorporating class weights and asymmetric terms to handle data imbalance.
• Evaluated TAFN performance on cross-subject classification of cognitive and motor imagery tasks using EEG-fNIRS data.

Main Results:

• TAFN achieved exceptional cross-subject accuracy: >99% for cognitive tasks and >97% for motor imagery (MI).
• The model demonstrated effectiveness in identifying subtle differences associated with epilepsy.
• Scalp topography analysis in MI tasks provided insights into epilepsy detection capabilities.

Conclusions:

• The TAFN model significantly outperforms traditional methods in high-precision brain activity detection.
• This technique shows promise for applications requiring the discernment of subtle neurological pattern differences, such as epilepsy and seizure detection.
• The developed TAFN offers a powerful tool for advancing the understanding and diagnosis of neurological conditions.