Manifold Feature Fusion with Dynamical Feature Selection for Cross-Subject Emotion Recognition

Area of Science:

• Neuroscience
• Computer Science
• Artificial Intelligence

Background:

• Affective computing decodes brain activity for emotional human-computer interaction.
• Individual neurophysiological differences pose challenges for generic emotion recognition (ER) systems.
• Cross-subject ER is difficult due to user-specific brain responses.

Purpose of the Study:

• To propose a novel feature selection method, manifold feature fusion and dynamical feature selection (MF-DFS), for cross-subject emotion recognition.
• To develop generalizable features sensitive to emotional variations using transfer learning.
• To enhance the accuracy and adaptability of emotion recognition systems.

Main Methods:

• Implemented manifold feature fusion and dynamical feature selection (MF-DFS) under a transfer learning framework.
• Utilized local geometrical information feature selection and domain adaptation based manifold learning.
• Validated MF-DFS on DEAP, MAHNOB-HCI, and SEED electroencephalography (EEG) databases using a leave-one-subject-out approach.

Main Results:

• MF-DFS achieved accuracies of 0.50-0.48 (DEAP) and 0.46-0.50 (MAHNOB-HCI) for arousal and valence dimensions.
• The SEED database yielded an accuracy of 0.40 for the valence dimension.
• MF-DFS significantly outperformed classical feature selection methods across multiple machine learning models.

Conclusions:

• The proposed MF-DFS method effectively enhances cross-subject emotion recognition by identifying generalizable features.
• MF-DFS demonstrates superior performance compared to traditional methods, paving the way for more adaptable affective computing systems.
• This approach addresses the challenge of individual variability in neurophysiological responses for improved emotion recognition.