Learning Sequential Variation Information for Dynamic Facial Expression Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Dynamic facial expression recognition (DFER) is crucial for human-computer interaction.
• Existing methods often struggle to effectively integrate information across different temporal scales in video sequences.
• Capturing both fine-grained frame-level details and long-range temporal dependencies remains a challenge.

Purpose of the Study:

• To introduce a novel multiscale sequence information fusion (MSSIF) method for DFER.
• To improve the accuracy and robustness of facial expression recognition in videos.
• To effectively handle short-term and long-term dependencies in dynamic facial expressions.

Main Methods:

• Developed a transformer-based architecture for hierarchical feature fusion.
• Integrated features from individual frames, subsequences, and entire video sequences.
• Employed deep feature extraction, self-attention mechanisms for intrasubsequence fusion, and intersubsequence fusion for long-term dynamics.

Main Results:

• Achieved high recognition accuracies on benchmark datasets: eNTERFACE'05 (60.1%), BAUM-1s (60.7%), and AFEW (58.8%).
• Demonstrated superior performance compared to existing methods in dynamic facial expression recognition.
• Validated the method's ability to manage both short-term and long-term dependencies effectively.

Conclusions:

• The MSSIF method offers a potent solution for accurate DFER.
• The hierarchical fusion strategy effectively leverages multiscale information for enhanced performance.
• The approach shows significant promise for real-world applications requiring nuanced facial expression analysis.