The limitations for expression recognition in computer vision introduced by facial masks

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Facial expression recognition (FER) leverages deep learning for high accuracy.
• Key facial regions for FER are the eyes and mouth.
• Facial masks, due to COVID-19 restrictions, severely impact current FER systems.

Purpose of the Study:

• To investigate the performance degradation of FER systems with mask occlusions.
• To compare the impact of mask occlusions versus eye occlusions on recognition accuracy.
• To assess the robustness of deep learning models in recognizing masked facial expressions.

Main Methods:

• Experiments were conducted on two public datasets.
• Analysis involved comparing recognition accuracy in occluded (mask, eye) versus non-occluded scenarios.
• State-of-the-art deep learning classifiers were evaluated.

Main Results:

• Deep classifiers showed a significant accuracy reduction (up to 50%) with facial masks.
• Eye occlusions also decreased performance, but less severely than masks.
• Masks represent a substantial limitation for current computer vision-based FER.

Conclusions:

• Facial masks pose a significant challenge to the accuracy of facial expression recognition systems.
• The performance drop caused by masks is more pronounced than that caused by eye occlusions.
• Further research is needed to develop robust FER solutions that can handle mask occlusions effectively.