Spectral-spatial feature fusion for real-time facial expression recognition

Area of Science:

• Computer Vision
• Affective Computing
• Machine Learning

Background:

• Facial Expression Recognition (FER) is crucial but current methods face computational challenges and limited feature extraction.
• Existing FER models struggle to capture subtle spatial and frequency-based discriminative features effectively.

Purpose of the Study:

• To propose SPAYOLO (Spectral-aware Perception and Aggregation YOLOv8), a novel FER network addressing computational costs and feature extraction limitations.
• To enhance FER performance by systematically modeling spatial and frequency features using a new module.

Main Methods:

• Developed the Spectral-aware Perception and Aggregation Module (SPAM) integrating Hierarchical Receptive Modeling (HRM) for spatial features and Frequency Enhancement Path (FEP) using Fast Fourier Transform (FFT) for frequency features.
• Implemented a Gated Attention Mechanism (GAM) for adaptive fusion of spatial and frequency features to improve stability.
• Utilized the YOLOv8 architecture as the base for the proposed SPAYOLO network.

Main Results:

• Achieved 70.74% accuracy on the FER2013 dataset and 67.88% on the AffectNet dataset.
• Demonstrated high computational efficiency, making it suitable for real-time facial expression recognition.
• Validated the effectiveness of hierarchical feature fusion and frequency-domain enhancement in FER.

Conclusions:

• SPAYOLO offers a computationally efficient and accurate solution for facial expression recognition.
• The proposed SPAM module effectively integrates spatial and frequency domain information for improved FER performance.
• Findings provide valuable insights for future research in computer vision and affective computing.