Cross-Layer Contrastive Learning of Latent Semantics for Facial Expression Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Facial expression recognition (FER) is crucial for human-computer interaction.
• Convolutional Neural Networks (CNNs) show promise in FER but struggle with inconsistent layer learning intensities.
• Shallow layers in CNNs often have insufficient feature representation compared to deeper layers.

Purpose of the Study:

• To propose a novel contrastive learning framework to address inconsistent learning intensities in CNNs for FER.
• To align feature semantics between shallow and deep layers for improved expression recognition.
• To enhance the learning intensity of shallow layer features through cross-layer contrastive learning.

Main Methods:

• A contrastive learning framework is proposed to align shallow and deep layer feature semantics.
• Cross-layer contrastive learning is employed to enhance the learning intensity of shallow layer features.
• An attention module is integrated to represent multi-scale features in a weight-adaptive manner.

Main Results:

• The proposed algorithm significantly enhances shallow layer feature learning intensity.
• Latent semantics in shallow and deep layer features are explored and aligned, improving fine-grained expression recognition.
• State-of-the-art performance achieved on RAF-DB (92.21%), FERPlus (89.50%), SFEW (62.82%), and AffectNet (65.29%).

Conclusions:

• The proposed framework effectively addresses the challenge of inconsistent learning intensities in CNNs for FER.
• Aligning shallow and deep layer features leads to more robust and accurate facial expression recognition.
• The method demonstrates superior performance across multiple challenging in-the-wild facial expression datasets.