TFE: A Transformer Architecture for Occlusion Aware Facial Expression Recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Facial expression recognition (FER) in uncontrolled environments is challenging due to unconstrained conditions.
• Existing deep learning methods struggle with occluded faces in real-world scenarios.

Purpose of the Study:

• To propose a transformer-based FER method (TFE) for robust recognition of facial expressions, especially on occluded faces.
• To enhance FER accuracy by adaptively focusing on discriminative and unoccluded facial regions.

Main Methods:

• Utilized a transformer architecture with a multi-head self-attention mechanism to process image patches.
• Integrated attention weights to create an attention map for guiding focus on important facial regions.
• Employed a decoder to reconstruct occluded facial regions for improved expression inference.

Main Results:

• The TFE method demonstrated improved recognition accuracy on both non-occluded and artificially occluded faces.
• Consistent performance gains were observed compared to state-of-the-art FER methods on AffectNet and RAF-DB datasets.
• Visualizations confirmed TFE's ability to focus on discriminative, unoccluded facial areas.

Conclusions:

• The proposed TFE method offers a robust solution for facial expression recognition in challenging, unconstrained environments.
• TFE's adaptive attention and reconstruction capabilities significantly enhance performance on occluded facial images.
• This approach advances the state-of-the-art in in-the-wild facial expression recognition.