A uniform human multimodal dataset for emotion perception and judgment.

Area of Science:

• Cognitive Neuroscience
• Social Psychology
• Neuroimaging

Background:

• Face perception is crucial for social interaction.
• Previous research often used single modalities, limiting holistic understanding.
• A need exists for integrated multimodal data in face perception research.

Purpose of the Study:

• To present a comprehensive multimodal dataset for facial emotion perception and judgment.
• To facilitate the study of face perception across different neuroscientific methods.
• To bridge the gap between human neuroimaging and neurophysiology literature.

Main Methods:

• Collected electroencephalography (EEG) data from 97 participants.
• Acquired functional magnetic resonance imaging (fMRI) data from 19 participants.
• Gathered single-neuron, eye-tracking, and behavioral data from neurosurgical patients, neurotypical individuals, and patient groups including those with Autism Spectrum Disorder (ASD) and amygdala lesions.

Main Results:

• The dataset encompasses diverse neuroscientific modalities (EEG, fMRI, single-neuron, eye-tracking, behavioral).
• All participants completed a standardized facial emotion perception task.
• The data enables cross-modal analysis of facial emotion processing.

Conclusions:

• This multimodal dataset offers a holistic approach to studying facial emotion perception.
• It highlights the importance of integrating diverse data streams for a complete understanding.
• The dataset is valuable for research on neuropsychiatric populations and fundamental cognitive processes.