Visual and auditory brain areas share a representational structure that supports emotion perception

Area of Science:

• Neuroscience
• Cognitive Science
• Psychology

Background:

• Emotionally expressive music and dance are globally ubiquitous.
• Shared sensory features across modalities may be represented similarly in distinct brain regions.
• This crossmodal similarity could facilitate the identification of emotionally relevant environmental cues.

Purpose of the Study:

• To investigate if visual and auditory brain areas share a common representational structure.
• To determine if crossmodal feature matching in music and animation elicits similar emotional perceptions.
• To explore the neural basis of shared emotional representations.

Main Methods:

• Created music and animation stimuli with crossmodally matched emotional features.
• Assessed participant perception of emotion in crossmodal stimuli.
• Utilized functional magnetic resonance imaging (fMRI) to examine brain activity during simultaneous music and animation presentation.

Main Results:

• Participants confirmed that specific features in music and movement consistently conveyed particular emotions.
• Brain imaging revealed similar representational structures in auditory and visual cortical areas for both music and animation.
• Shared representations in the posterior superior temporal cortex and early visual cortex supported emotion judgments and stimulus processing.

Conclusions:

• Crossmodally shared representations in the brain support the perception and comparison of emotional content across senses.
• These shared neural structures may explain the ease of perceiving emotions and the cross-cultural generalization of dynamic emotional expressions.
• Mutually reinforcing interactions between auditory and visual brain areas are facilitated by common representational frameworks.