Emotion Recognition From Multimodal Physiological Signals Using a Regularized Deep Fusion of Kernel Machine

Area of Science:

• Physiology
• Computer Science
• Machine Learning

Background:

• Physiological signals are increasingly used for emotion recognition in human-computer interaction.
• Challenges include the complexity of emotions and individual differences in physiological responses, necessitating reliable models.

Purpose of the Study:

• To propose a regularized deep fusion framework for robust emotion recognition using multimodal physiological signals.
• To improve subject-independent emotion recognition performance.

Main Methods:

• Extracting effective features from various physiological signals.
• Constructing ensemble dense embeddings using kernel matrices.
• Utilizing a deep network for task-specific representations and a global fusion layer with regularization to synchronize optimization.

Main Results:

• The proposed framework significantly improves subject-independent emotion recognition compared to single-modal approaches and other fusion methods.
• Experiments on benchmark datasets validate the framework's effectiveness.
• Data visualization confirms enhanced class-separability power in the final fused representation.

Conclusions:

• The regularized deep fusion framework offers a superior approach for emotion recognition from multimodal physiological signals.
• This method addresses challenges posed by individual differences and emotional complexity.
• The framework demonstrates potential for more intelligent human-computer interaction systems.