Visually-Inspired Multimodal Iterative Attentional Network for High-Precision EEG-Eye-Movement Emotion Recognition.

Area of Science:

• Affective Computing
• Artificial Intelligence
• Biomedical Signal Processing

Background:

• Affective computing aims to recognize human emotions using AI.
• Integrating multimodal data like EEG and eye movements can improve emotion recognition accuracy.
• Existing methods face challenges with data sparsity and feature compatibility.

Purpose of the Study:

• To develop a novel multimodal framework for enhanced emotion recognition.
• To synergistically integrate electroencephalography (EEG) and eye-movement (EM) features.
• To improve the reliability and accuracy of affective computing systems.

Main Methods:

• EEG Feature Encoder (EFE) using a convolutional architecture for neural pattern extraction.
• EM Feature Encoder (EMFE) employing a Kolmogorov-Arnold Network (KAN) for sparse EM data.
• Multimodal Iterative Attentional Feature Fusion (MIAFF) module with Hierarchical Channel Attention (HCAM) for feature integration.

Main Results:

• Achieved leading-edge accuracy on SEED (3-class) and SEED-IV (4-class) emotion recognition benchmarks.
• Demonstrated the effectiveness of biomimetic encoding and iterative attention in multimodal fusion.
• Highlighted the potential of the framework for advanced brain-computer interface applications.

Conclusions:

• The proposed multimodal framework significantly enhances emotion recognition accuracy by integrating EEG and EM data.
• Biomimetic encoding and iterative attention are powerful techniques for affective computing.
• This research paves the way for next-generation brain-computer interfaces in various fields.