CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Area of Science:

• Affective computing
• Human-computer interaction
• Biomedical signal processing

Background:

• Emotion recognition is crucial for personalized video content, but current methods are limited to single emotions or lab settings.
• Wearable physiological signals offer a promising avenue for continuous, real-world emotion detection.

Purpose of the Study:

• To develop and validate a correlation-based emotion recognition algorithm (CorrNet) for fine-grained valence and arousal (V-A) detection using wearable physiological signals.
• To evaluate CorrNet's performance in both indoor-desktop and outdoor-mobile environments.

Main Methods:

• CorrNet utilizes intra-modality and correlation-based features from physiological signals (electrodermal activity, heart rate).
• The algorithm was tested on the CASE (indoor-desktop) and MERCA (outdoor-mobile) datasets, the latter collected using a smart wristband and wearable eyetracker.
• Subject-independent binary classification (high-low) was employed for V-A recognition.

Main Results:

• CorrNet achieved promising subject-independent binary classification accuracies: 76.37% (valence) and 74.03% (arousal) on CASE, and 70.29% (valence) and 68.15% (arousal) on MERCA.
• Optimal instance segment lengths for recognition were found to be between 1-4 seconds.
• Comparable accuracies were observed between laboratory-grade and wearable sensors, even at low sampling rates (≤64 Hz).

Conclusions:

• CorrNet demonstrates the feasibility of real-time, wearable-based emotion recognition for personalized video experiences.
• Wearable sensors provide comparable accuracy to lab-grade equipment for V-A recognition.
• The study highlights the impact of neutral V-A labels on recognition performance, suggesting careful consideration during annotation.