Dynamic Facial Expression Recognition Under Partial Occlusion With Optical Flow Reconstruction

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Facial expression recognition (FER) is vital for many applications.
• Partial facial occlusion significantly challenges existing FER methods.
• Current occlusion handling methods focus on texture or geometry, overlooking movement similarity.

Purpose of the Study:

• To propose a novel method for reconstructing occluded facial parts in the optical flow domain for improved FER.
• To leverage the similarity of facial movements across individuals for reconstruction.
• To establish a new protocol for reproducible FER research with occlusions.

Main Methods:

• Developed an auto-encoder with skip connections to reconstruct occluded facial movements.
• Operated directly in the optical flow domain for movement reconstruction.
• Generated synthetic occlusions on controlled datasets (CK+) for validation.

Main Results:

• The proposed method effectively reduces the accuracy gap between occluded and unoccluded facial expression recognition.
• Demonstrated the efficacy of reconstructing facial movement for FER.
• Outperformed existing state-of-the-art approaches in occluded scenarios.

Conclusions:

• Reconstructing facial movement in the optical flow domain is a promising approach for robust FER under occlusion.
• The novel auto-encoder method enhances recognition accuracy by addressing occlusion challenges.
• The proposed experimental protocol facilitates future reproducible research in occluded FER.