Brain and Body Emotional Responses: Multimodal Approximation for Valence Classification.

Area of Science:

• Neuroscience
• Psychology
• Biomedical Engineering

Background:

• Emotion recognition research often lacks integration between psychological theory and engineering applications.
• Understanding the interplay between central and peripheral nervous system signals is crucial for affective computing.
• Existing methods for emotion recognition using physiological signals have limitations in precision and functional application.

Purpose of the Study:

• To develop a computational model for emotion recognition in the valence dimension by studying the psychobiology of central and peripheral nervous systems.
• To investigate the effectiveness of electroencephalography (EEG), electrocardiography (ECG), and skin temperature for emotion classification.
• To determine if a multimodal approach improves emotion recognition compared to individual modalities.

Main Methods:

• Collected EEG, ECG, and skin temperature data from 24 subjects during emotional tasks.
• Individually evaluated each physiological signal for characteristic emotion patterns.
• Performed feature selection for each modality and applied classification algorithms.
• Analyzed results, including comparisons between central and peripheral responses and by sex.

Main Results:

• Individual physiological signals (EEG, ECG, skin temperature) showed distinct patterns for positive and negative emotions.
• Electroencephalography (EEG) alone provided the most effective classification of emotional valence.
• The multimodal approach combining EEG, ECG, and skin temperature did not outperform EEG alone.
• Sex-based analysis revealed notable differences in peripheral nervous system responses to emotional stimuli between males and females.

Conclusions:

• Emotion recognition is feasible using both central (EEG) and peripheral (ECG, skin temperature) nervous system signals.
• Electroencephalography (EEG) is a highly effective modality for recognizing emotional valence.
• Integrating peripheral signals does not enhance emotion recognition accuracy beyond EEG.
• Sex influences emotional processing, particularly at the peripheral nervous system level.