FEELing (key)Pressed: Implicit Touch Pressure Bests Brain Activity for Modeling Emotion Dynamics in the Space Between...

Area of Science:

• Affective computing
• Human-computer interaction
• Computational neuroscience

Background:

• Real-time emotion estimation faces challenges due to the dynamic and individual nature of human emotions.
• Existing models often oversimplify emotions as discrete states, neglecting their continuous transitions and complexity.
• A need exists for advanced methods that capture the nuanced evolution of emotional experiences.

Purpose of the Study:

• To develop and validate a multi-resolution emotion self-reporting procedure for capturing dynamic emotional states.
• To compare the efficacy of different modalities (brain activity and keypress force) for real-time emotion classification.
• To introduce the FEEL (Force, EEG and Emotion-Labelled) dataset for advancing emotion recognition research.

Main Methods:

• Trained participant-dependent hierarchical models on contextualized individual experiences.
• Collected data on brain activity (EEG) and keypress force during emotionally engaging tasks (videogame play).
• Utilized a multi-resolution self-reporting procedure to label emotion transitions along a Stressed-Relaxed scale.

Main Results:

• Achieved F1-scores ranging from 0.44 to 0.82, significantly outperforming chance (F1=0.22).
• Identified pressure-based features from keypress force as a more informative modality for classifying emotion evolution in stressful contexts.
• Demonstrated the convenience and lower intrusiveness of keypress force data collection compared to brain activity.

Conclusions:

• Participant-dependent hierarchical models effectively capture dynamic emotional transitions.
• Keypress force offers a practical and informative modality for real-time emotion recognition, especially in dynamic stress scenarios.
• The open-sourced FEEL dataset provides valuable resources for future research in affective computing and emotion estimation.