Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data.

Area of Science:

• Physiological computing
• Affective computing
• Machine learning for healthcare

Background:

• Wearable sensors offer potential for mobile health and automotive applications.
• Current wearable technology struggles to extract complex affective states like emotions.
• Advanced methods are needed for accurate physiological information extraction.

Purpose of the Study:

• To investigate end-to-end multimodal deep learning for affect recognition.
• To compare the performance of ten deep learning architectures on diverse physiological datasets.
• To evaluate the suitability of different deep learning models for recognizing emotional and stress states.

Main Methods:

• Utilized ten end-to-end deep learning architectures for time-series classification.
• Enhanced architectures to process multi-sensor data with varying sampling frequencies.
• Employed Bayesian optimization for rigorous hyperparameter tuning across models.

Main Results:

• Model performance varied based on the intensity of the induced physiological response.
• Stress recognition (Trier Social Stress Test) was more successful than emotion recognition from affective content.
• Convolutional Neural Network (CNN)-based architectures demonstrated greater suitability than Long Short-Term Memory (LSTM)-based architectures.

Conclusions:

• End-to-end deep learning shows potential for affect recognition using wearable physiological data.
• CNNs appear more effective than LSTMs for affect recognition from physiological signals.
• Future research should consider the intensity of affective stimuli for improved model performance.