Occlusion aware facial expression recognition using CNN with attention mechanism

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Facial expression recognition in unconstrained environments is challenging due to occlusions.
• Existing methods perform poorly on partially occluded faces, common in real-world settings.

Purpose of the Study:

• To propose an attention-based Convolutional Neural Network (ACNN) for robust facial expression recognition.
• To develop a framework that can identify and focus on unoccluded facial regions for improved accuracy.

Main Methods:

• Introduced an end-to-end learning framework, ACNN, incorporating an attention mechanism.
• Proposed a Gate Unit to adaptively weigh facial region representations based on unobstructed-ness and importance.
• Developed two ACNN versions: patch-based (pACNN) and global-local (gACNN).

Main Results:

• ACNNs significantly improved recognition accuracy on both occluded and non-occluded faces.
• Visualizations confirmed ACNNs shift attention from occluded to unoccluded facial areas.
• Outperformed state-of-the-art methods on in-the-lab datasets under cross-dataset evaluation.

Conclusions:

• The proposed ACNN framework enhances facial expression recognition accuracy in the presence of occlusions.
• ACNNs demonstrate superior performance in handling unconstrained facial expression recognition tasks.
• The attention mechanism is crucial for focusing on relevant facial features despite occlusions.