Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors

Area of Science:

• Affective Computing
• Human-Computer Interaction
• Biomedical Engineering

Background:

• Emotion recognition traditionally uses non-immersive stimuli.
• Immersive Virtual Environments (IVEs) offer novel possibilities for affective state elicitation.
• The Circumplex Model of Affects provides a framework for understanding emotional dimensions.

Purpose of the Study:

• To develop an emotion recognition system for affective states evoked through IVEs.
• To investigate the use of electroencephalography (EEG) and electrocardiography (ECG) signals for emotion detection.
• To validate IVEs as a tool for eliciting and recognizing emotions.

Main Methods:

• Designed four virtual rooms to elicit different arousal-valence states.
• Recorded EEG and ECG data from sixty participants.
• Extracted features quantifying neural and cardiovascular dynamics, feeding them into a Support Vector Machine classifier.

Main Results:

• Achieved 75.00% accuracy in predicting arousal.
• Achieved 71.21% accuracy in predicting valence.
• Demonstrated the efficacy of biosignal analysis within IVEs for emotion recognition.

Conclusions:

• IVEs are effective for eliciting and automatically recognizing diverse emotional states.
• Neural and cardiac dynamics can be reliably used for emotion prediction in immersive settings.
• This research has potential applications in Architecture, Health, Education, and Videogames.