EIMNet: An EEG and iEEG-Fused Interactive Modality Network for Accurate Memory State Prediction during Working Memory...

Area of Science:

• Neuroscience
• Biomedical Engineering
• Cognitive Science

Background:

• Brain-Computer Interface (BCI) research increasingly relies on multimodal integration for effective feature extraction.
• Working memory (WM) tasks present challenges for accurately decoding cognitive states.
• Existing BCI models may not fully leverage the complementary information from different neuroimaging modalities.

Purpose of the Study:

• To introduce EIMNet, a novel cross-modality fusion model for enhanced feature representation in BCI.
• To investigate the efficacy of integrating electroencephalography (EEG) and intracranial electroencephalography (iEEG) for WM tasks.
• To improve the prediction accuracy of memory-related cognitive effects using multimodal BCI.

Main Methods:

• Developed EIMNet, a fusion model inspired by phase-amplitude coupling to enable EEG-iEEG interaction.
• Utilized ablation experiments to identify key factors influencing decoding performance (e.g., interaction factor, frequency bands, data augmentation).
• Evaluated EIMNet's effectiveness in enhancing decoding accuracy for WM tasks.

Main Results:

• EIMNet demonstrated enhanced representation of task-related features by enabling EEG-iEEG interaction.
• Ablation studies confirmed the significant impact of interaction factor selection, frequency band splitting, and data augmentation.
• The integrated EEG and iEEG approach using EIMNet significantly improved decoding accuracy for WM tasks.

Conclusions:

• EIMNet effectively integrates EEG and iEEG data, leading to improved decoding performance in WM tasks.
• The findings highlight the importance of multimodal fusion and specific model parameters for BCI applications.
• EIMNet shows promise for advancing research in memory and attention-related cognitive functions.