Social-affective features drive human representations of observed actions

Area of Science:

• Cognitive Neuroscience
• Social Psychology
• Computer Vision

Background:

• Human perception involves processing complex visual scenes with actions.
• Understanding how the brain extracts relevant features from observed actions is crucial.
• Naturalistic actions present a rich source of visual, action-related, and social-affective information.

Purpose of the Study:

• To investigate which features (visual, action, social-affective) drive similarity judgments of naturalistic actions.
• To determine how these features are processed temporally in the brain during action observation.

Main Methods:

• Curated two large-scale naturalistic video datasets of everyday actions.
• Conducted behavioral experiments to estimate perceived similarity and quantify features.
• Utilized cross-validated variance partitioning and electroencephalography (EEG) to analyze feature importance and neural correlates.

Main Results:

• Social-affective features significantly predicted behavioral similarity judgments, outperforming visual and action features.
• EEG data revealed a sustained neural response correlated with behavioral similarity.
• Visual, action, and social-affective features mapped to early, intermediate, and late neural processing stages, respectively.

Conclusions:

• Social-affective features play a critical role in the perception of naturalistic actions.
• The brain processes these social-affective aspects of actions at a later stage compared to visual and motor information, following a temporal gradient.