Decoupled Doubly Contrastive Learning for Cross-Domain Facial Action Unit Detection

Area of Science:

• Computer Vision
• Machine Learning
• Biometrics

Background:

• Current vision-based facial action unit (AU) detection methods struggle with domain variations.
• Cross-domain AU detection remains an under-explored research area, limiting real-world applicability.

Purpose of the Study:

• To develop a robust cross-domain facial AU detection method.
• To learn a purified AU representation semantically aligned across different domains.
• To enable intuitive control over cross-domain facial image synthesis.

Main Methods:

• Propose Decoupled Doubly Contrastive Adaptation (D2CA) to decompose latent representations into AU-relevant and AU-irrelevant components.
• Employ feature decoupling by assessing synthesized faces with modified AU or domain attributes.
• Utilize doubly contrastive learning (image and feature-level) to strengthen decoupling, especially with limited data diversity.

Main Results:

• D2CA successfully decouples AU and domain factors, enabling visually pleasing cross-domain synthesized facial images.
• The method achieves significant performance improvements over state-of-the-art cross-domain AU detection approaches.
• An average F1 score improvement of 6%-14% was observed across various cross-domain scenarios.

Conclusions:

• D2CA provides an effective framework for learning dedicated separation of AU-relevant and domain-relevant factors.
• The approach enhances cross-domain AU detection accuracy and facial image synthesis capabilities.
• This work addresses a critical limitation in current facial expression recognition systems.