Affect differentially modulates brain activation in uni- and multisensory body-voice perception

Area of Science:

• Neuroscience
• Cognitive Psychology
• Sensory Processing

Background:

• Emotion perception relies on integrating information from multiple senses.
• Previous research focused on classical multisensory integration, neglecting other interactions like crossmodal prediction.
• The role of crossmodal prediction in perceiving emotions remains unclear.

Purpose of the Study:

• To investigate the role of crossmodal prediction in multisensory emotion perception.
• To differentiate between classical integration and predictive mechanisms in emotion processing.
• To examine brain activation patterns during the perception of emotional and neutral body and vocal expressions.

Main Methods:

• Utilized video clips featuring emotional and neutral body and vocal expressions.
• Employed functional magnetic resonance imaging (fMRI) to measure brain activity.
• Analyzed brain activation in response to different types of emotional and neutral multisensory stimuli.

Main Results:

• Emotional multisensory expressions increased activation in the bilateral fusiform gyrus (FFG).
• Neutral expressions, compared to emotional ones, enhanced activation in the bilateral middle temporal gyrus (MTG) and posterior superior temporal sulcus (STS).
• Neutral stimuli activated classical multisensory areas, whereas emotional stimuli engaged unisensory visual processing areas.

Conclusions:

• Multisensory emotion perception involves at least two distinct mechanisms: classical multisensory integration and crossmodal prediction.
• Neutral expressions appear to rely on classical multisensory integration.
• Emotional expressions seem to involve crossmodal prediction, potentially driven by visual information predicting auditory input, especially for salient stimuli.