Physiological Sensors Based Emotion Recognition While Experiencing Tactile Enhanced Multimedia

Area of Science:

• Affective computing
• Human-computer interaction
• Biomedical engineering

Background:

• Emotion recognition enhances user understanding and behavior analysis.
• Physiological sensors (EEG, PPG, GSR) are used for emotion detection.
• Current methods engage limited human senses.

Purpose of the Study:

• To recognize human emotions using physiological signals from tactile-enhanced multimedia.
• To provide users with enhanced real-world sensations during multimedia engagement.
• To explore the fusion of multiple physiological signals for improved emotion classification.

Main Methods:

• Selected videos synchronized with fan/heater for tactile effects (cold/hot air).
• Recorded electroencephalography (EEG), photoplethysmography (PPG), and galvanic skin response (GSR) signals.
• Applied Savitzky-Golay filter for signal pre-processing, extracted features, and used K-nearest neighbor classifier.

Main Results:

• Photoplethysmography (PPG) features yielded the highest individual accuracy (78.57%).
• Fusion of EEG, GSR, and PPG features improved emotion classification accuracy to 79.76%.
• Tactile-enhanced multimedia significantly impacted physiological responses for emotion recognition.

Conclusions:

• Physiological signal fusion offers a promising approach for advanced emotion recognition.
• Tactile-enhanced multimedia can enrich user experience and provide richer affective data.
• This study advances affective computing by integrating multi-sensory and physiological data.