Predicting affective appraisals from facial expressions and physiology using machine learning.

Area of Science:

• Psychophysiology
• Affective Science
• Machine Learning in Psychology

Background:

• Understanding the physiological underpinnings of emotion is crucial in affective science.
• Previous research has linked physiological responses to emotional states, but the precise relationships with cognitive appraisal dimensions remain less understood.

Purpose of the Study:

• To explore the interrelations between various appraisal dimensions and multiple physiological signals.
• To determine the predictability of appraisal dimensions from physiological features using machine learning.
• To investigate the nature (linear vs. nonlinear) of the relationship between physiological signals and appraisal dimensions.

Main Methods:

• 157 participants watched 10 emotionally charged videos.
• Facial electromyography (EMG), electrodermal activity, and heart rate variability were recorded.
• 134 features were extracted from physiological data, and machine learning algorithms were benchmarked.
• Accumulated local effects plots were used to analyze feature-appraisal relationships.

Main Results:

• 13 out of 21 appraisal dimensions were significantly predictable from physiological features (positive R²).
• Intrinsic pleasantness showed the highest predictability (R² = .407).
• The relationship between physiological signals and appraisals was predominantly nonlinear.
• Facial EMG, electrodermal activity, and heart rate variability contributed differently to predicting appraisals.

Conclusions:

• Physiological signals contain information that can predict cognitive appraisal dimensions of emotion.
• The complex, nonlinear nature of these relationships necessitates advanced analytical techniques.
• Specific physiological channels offer unique insights into distinct appraisal processes.