Cross-modal deep generative models reveal the cortical representation of dancing

Area of Science:

• Neuroscience
• Cognitive Science
• Artificial Intelligence

Background:

• Dance is a complex, holistic art form offering insights into human cognition and emotion.
• Previous research has lacked fine-grained, quantitative analyses of how dance information is processed in the brain.

Purpose of the Study:

• To investigate how cross-modal features of dance are represented in the brain.
• To compare the explanatory power of generative model features versus low-level features for dance-evoked brain activity.
• To explore how different emotions evoked by dance relate to distinct neural patterns and the role of expertise.

Main Methods:

• Utilized a cross-modal deep generative model to extract features from dance.
• Applied functional magnetic resonance imaging (fMRI) to record brain activity while participants viewed dance clips.
• Employed encoding models to simulate and quantify neural responses to dance features.

Main Results:

• Cross-modal features from the generative model significantly explained brain activity better than low-level motion and audio features.
• Encoding models revealed distinct neural patterns corresponding to dances eliciting different emotions.
• Expert dancers showed broader neural explanation by dance features compared to novices, alongside greater individual variability.

Conclusions:

• Cross-modal representations derived from generative models effectively link to naturalistic neuroimaging data.
• The study clarifies the neural basis of aesthetic and emotional experiences in dance, highlighting the interplay of motion, music, and expertise.