A multi-granularity feature fusion approach with attention for facial expression recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Facial expression recognition (FER) methods struggle with fine-grained local variations and limited feature representation.
• Existing approaches often fail to capture the nuances of subtle facial changes crucial for accurate emotion detection.

Purpose of the Study:

• To propose a novel FER method, MGFA, that addresses limitations in sensitivity and feature representation.
• To enhance FER performance by effectively integrating global and local facial features through multi-granularity fusion and attention.

Main Methods:

• Developed MGFA (Multi-granularity Feature fusion with Attention) for FER.
• Utilized a Global Multi-scale Feature Extraction Module (GMFEM) with channel attention.
• Employed a Local Multi-granularity Feature Extraction Module (LMFEM) with spatial segmentation and multi-scale attention.
• Integrated features using a Cross-Fusion Module (CFM) to capture local and global details.

Main Results:

• MGFA demonstrated significant improvements in accuracy and robustness on three public FER datasets.
• The proposed method effectively enhances facial expression feature representation by combining multi-scale global and local information.
• Attention mechanisms and multi-granularity fusion proved effective in capturing subtle facial cues.

Conclusions:

• The MGFA method offers a superior approach to FER compared to existing techniques.
• Effective fusion of multi-granularity features with attention mechanisms is key to advancing FER.
• The proposed model shows strong potential for real-world applications requiring precise emotion recognition.