An improved multi-input deep convolutional neural network for automatic emotion recognition.

Area of Science:

• Computational neuroscience
• Affective computing
• Machine learning

Background:

• Current emotion recognition models using physiological signals often rely on single data types.
• This approach overlooks crucial inter-correlations between different physiological signal modalities.
• Integrating multi-modal physiological data can enhance the accuracy of emotion recognition.

Purpose of the Study:

• To design and evaluate a novel end-to-end multi-input deep convolutional neural network (MI-DCNN).
• To leverage multi-modal physiological signals for simultaneous feature extraction and emotion classification.
• To improve upon existing single-modal deep convolutional neural network (DCNN) approaches for emotion recognition.

Main Methods:

• An emotion elicitation experiment was conducted with 52 participants.
• Physiological signals including electrocardiography (ECG), electrodermal activity (EDA), and respiratory activity (RSP) were collected.
• A novel MI-DCNN architecture was developed and validated against traditional machine learning methods and single-input DCNNs on multiple datasets.

Main Results:

• The MI-DCNN demonstrated significant improvements over traditional machine learning methods for both arousal and valence recognition (p < 0.01).
• Baseline accuracy for arousal and valence using traditional methods were 62.9% and 60.3%, respectively.
• The proposed MI-DCNN effectively integrates multi-modal physiological data, outperforming single-input DCNNs.

Conclusions:

• The developed MI-DCNN architecture is effective for emotion recognition using multi-modal physiological signals.
• Integrating multiple physiological signal modalities enhances the performance of deep learning models for emotion detection.
• This approach offers a promising direction for more accurate and robust emotion recognition systems.