Semantic Neighborhood-Aware Deep Facial Expression Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Affective Computing

Background:

• Facial Expression Recognition (FER) consistency is crucial, as minor input changes should yield minor output changes.
• Existing FER datasets suffer from data imbalance, noise, and insufficient data, compromising recognition consistency and performance.
• Current FER methods often overlook output stability concerning semantic input perturbations.

Purpose of the Study:

• To enhance the consistency and stability of Facial Expression Recognition (FER) models.
• To address data imbalance, noise, and data scarcity issues in FER datasets.
• To develop a method that accounts for neighborhood smoothness alongside prediction accuracy.

Main Methods:

• A novel method is proposed to formulate semantic perturbation for input data.
• The method incorporates neighborhood smoothness into the FER model's training objective.
• Unreliable samples are identified and selected during training to mitigate their negative impact.

Main Results:

• The proposed method demonstrates significant effectiveness in improving FER consistency.
• State-of-the-art results are achieved on the AffectNet dataset, the largest in-the-wild FER database.
• Performance is improved by 30% compared to existing state-of-the-art methods, approaching an upper limit.

Conclusions:

• The novel approach successfully enhances the stability and accuracy of facial expression recognition.
• Addressing data quality issues and incorporating output smoothness are key to advancing FER.
• The method offers a promising direction for more robust and reliable facial expression analysis.