Emotion analysis and recognition in 3D space using classifier-dependent feature selection in response to tactile...

Area of Science:

• Human-Computer Interaction
• Neuroscience
• Multimedia Systems

Background:

• Traditional media engages limited senses (vision, hearing).
• Multi-sensory media aims for immersive experiences by integrating touch, smell, and taste.
• Tactile-enhanced audio-visual content adds touch to visual and auditory stimuli for deeper user engagement.

Purpose of the Study:

• To investigate the impact of tactile-enhanced audio-visual content on user emotional states.
• To analyze electroencephalogram (EEG) data in response to multi-sensory media.
• To develop and evaluate a classification method for emotional states (valence, arousal, dominance) evoked by this content.

Main Methods:

• Collected EEG data from participants interacting with tactile-enhanced and traditional audio-visual content.
• Utilized a Self-Assessment Manikin scale to label emotional dimensions: valence, arousal, and dominance.
• Employed a classifier-dependent feature selection approach with three distinct classifiers to categorize emotional states.

Main Results:

• Established statistically significant differences (95% CI) in arousal and dominance between tactile-enhanced and traditional media.
• Achieved high classification accuracies: 75% for valence, 73.8% for arousal, and 75% for dominance.
• Demonstrated superior performance over previous emotion recognition studies in accuracy and F-score.

Conclusions:

• Tactile-enhanced audio-visual content significantly impacts user emotional dimensions, particularly arousal and dominance.
• The proposed classification methodology effectively identifies emotional states evoked by multi-sensory media.
• This research advances emotion recognition in enhanced multimedia, paving the way for more immersive user experiences.