Cross-domain AU Detection: Domains, Learning Approaches, and Measures

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Affective Computing

Background:

• Facial action unit (AU) detection models excel within their training domain.
• Generalizability of these models to new, unseen domains remains a significant challenge.
• Existing research has limitations in evaluating cross-domain performance comprehensively.

Purpose of the Study:

• To investigate the cross-domain transferability of facial action unit (AU) detectors.
• To compare the performance of deep learning (CNN) and shallow learning (SVM) approaches in cross-domain settings.
• To identify factors influencing the generalizability of AU detection models.

Main Methods:

• Reviewed literature on cross-domain transfer learning for AU detection.
• Conducted experiments using two large, well-annotated public databases (Expanded BP4D+ and GFT).
• Evaluated both Convolutional Neural Network (CNN) and Support Vector Machine (SVM) based AU detection methods.

Main Results:

• AU detector performance significantly decreased when transferring to new domains.
• Performance drops were more substantial for the GFT database compared to Expanded BP4D+.
• Deep learning (CNN) models demonstrated better cross-domain generalizability than shallow learning (SVM) models.
• Performance degradation in some cases fell below usability thresholds for behavioral research.

Conclusions:

• Cross-domain transfer of AU detectors is challenging, with performance degradation observed for both deep and shallow methods.
• Deep learning approaches and training on more diverse datasets show promise for improving generalizability.
• Caution is necessary when deploying AU classifiers across different domains due to performance variability.