Facial expression recognition in virtual reality environments: challenges and opportunities

Area of Science:

• Computer Science
• Human-Computer Interaction
• Psychology

Background:

• Facial emotion recognition (FER) is crucial for understanding user experience in virtual reality (VR).
• Existing FER systems face challenges in accurately detecting nuanced emotions within dynamic VR environments.

Purpose of the Study:

• To evaluate the effectiveness of a novel facial emotion recognition system using MobileNet V2 in a virtual reality setting.
• To identify specific emotions that are accurately recognized and those that pose challenges for the system.

Main Methods:

• A lightweight convolutional neural network, MobileNet V2, was employed for emotion detection.
• The system was tested on 15 university students interacting within a virtual reality environment.
• Performance was assessed based on recognition rates for various facial expressions.

Main Results:

• High recognition rates were achieved for "Neutral", "Happiness", "Sadness", and "Surprise" emotions.
• The system demonstrated lower accuracy for "Anger" and "Fear", frequently misclassifying them as "Neutral".
• Discrepancies were potentially linked to overlapping facial cues, limited training data, and device precision.

Conclusions:

• Facial emotion recognition technology is viable for virtual reality applications.
• Future development requires model enhancements, superior hardware, and a comprehensive approach to emotion detection.
• Further research is needed to improve accuracy for complex emotions in VR.