Cross-Stimulus Transfer Learning: Enhancing Emotion Recognition from Visual-Auditory to Olfactory Perception

Area of Science:

• Neuroscience
• Machine Learning
• Affective Computing

Background:

• Olfactory stimuli offer unique neural pathways to the amygdala and hippocampus, crucial for emotion processing.
• Emotion research has predominantly focused on visual-auditory stimuli, leaving olfactory-induced emotions underexplored.

Purpose of the Study:

• To investigate the potential of cross-stimulus transfer learning for emotion recognition using olfactory and visual electroencephalography (EEG) data.
• To address the gap in understanding olfactory-induced emotions compared to other sensory modalities.

Main Methods:

• A cross-stimulus transfer learning task was designed using two EEG datasets: one olfactory-induced and the SEED dataset (video-induced).
• Domain adaptation (DA) and domain generalization (DG) techniques were employed to mitigate stimulus type differences and individual variability.
• A Transformer-based Domain-Adversarial Neural Network (DANN) was utilized for emotion recognition.

Main Results:

• The Transformer-based DANN achieved 50.7% cross-subject and 44.0% cross-stimulus emotion recognition accuracy.
• These results outperformed conventional methods in cross-stimulus emotion recognition tasks.
• The study confirmed the feasibility of transferring emotion recognition models across different sensory stimuli.

Conclusions:

• Cross-stimulus transfer learning is a viable approach for emotion recognition, even with olfactory stimuli.
• This research provides experimental support for identifying stimulus-independent emotion patterns.
• Findings pave the way for more robust and versatile brain-computer interfaces for emotion detection.