Virtual reality facial emotion recognition in social environments: An eye-tracking study

Area of Science:

• Psychology
• Computer Science
• Neuroscience

Background:

• Virtual reality (VR) provides realistic social stimuli for emotion recognition assessment and training.
• Novel VR tasks were developed to evaluate emotion recognition compared to traditional photo and video methods.
• The study investigated factors influencing recognition and explored visual attention within VR.

Purpose of the Study:

• To compare emotion recognition accuracy across virtual reality (VR), video, and photo tasks.
• To identify covariates affecting emotion recognition performance.
• To explore patterns of visual attention during VR-based emotion recognition.

Main Methods:

• 100 healthy participants completed photo, video, and VR emotion recognition tasks.
• In VR, participants rated emotions of virtual characters in a simulated street environment.
• Eye-tracking data was collected during the VR task to analyze visual attention.

Main Results:

• VR emotion recognition accuracy (75%) was similar to photo and video tasks.
• Disgust and happiness recognition were less accurate in VR, while surprise and anger recognition were better compared to video.
• Participants focused attention longer on the eye and nose regions than the mouth.

Conclusions:

• Immersive VR tasks are suitable for both training and assessing emotion recognition.
• VR allows for controllable avatars in ecologically relevant environments.
• Validated VR emotion recognition tasks hold potential for clinical applications.