End-to-End Training for Compound Expression Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human emotions are complex and often expressed through compound facial expressions.
• Current research primarily focuses on basic emotions, neglecting the nuances of real-life emotional states.
• Accurate recognition of compound emotions is crucial for advancing human-computer interaction.

Purpose of the Study:

• To develop an effective method for recognizing compound facial expressions in the wild.
• To explore the integration of spatial and frequency domain features for enhanced emotion recognition.
• To investigate the benefits of model ensembling for improving the accuracy of compound emotion recognition.

Main Methods:

• Utilized deep learning models for extracting appearance features, including ResNet50 fine-tuned on facial expression databases and wavelet transforms (DWT2, WAVEDEC2).
• Employed geometric feature extraction using densesift operators for facial key points, followed by training deep SAE, stacked LSTM, and Sequence-to-Sequence models.
• Implemented end-to-end joint training by ensembling appearance and geometric feature models for improved recognition performance.

Main Results:

• Achieved a recognition rate of 66.97% on the RAF Compound database for compound expression recognition.
• Demonstrated that integrating models capturing different information (appearance and geometric) improves recognition accuracy.
• Validated the effectiveness of end-to-end joint training for compound emotion recognition.

Conclusions:

• The proposed framework effectively recognizes compound facial expressions by combining diverse feature representations.
• Model ensembling and end-to-end training are key strategies for enhancing the performance of complex emotion recognition systems.
• This research contributes to more sophisticated human-computer emotional communication.