Learning Facial Action Units from Web Images with Scalable Weakly Supervised Clustering

Area of Science:

• Computer Vision
• Machine Learning
• Biometrics

Background:

• Facial Action Unit (AU) detection is crucial for understanding human emotions.
• Existing methods often require fully annotated datasets, limiting scalability.
• Leveraging large, freely available web images with noisy annotations presents a significant challenge.

Purpose of the Study:

• To develop a scalable weakly supervised clustering approach for learning facial action units (AUs).
• To utilize web images with inaccurate annotations for AU detection.
• To create a method that can effectively train AU classifiers without extensive manual annotation.

Main Methods:

• A weakly-supervised spectral algorithm was developed to learn an embedding space coupling image appearance and semantics.
• The algorithm features efficient gradient updates and a stochastic extension for scalability to large datasets.
• Rank-order clustering was employed on the learned embedding space to group similar images for re-annotation and classifier training.

Main Results:

• Learned annotations achieved an average of 91.3% agreement with human annotations for 7 common AUs on the EmotioNet dataset.
• Classifiers trained with re-annotated data performed comparably to, and sometimes surpassed, supervised Convolutional Neural Network (CNN) based methods.
• The method provides intuitive outlier and noise pruning capabilities.

Conclusions:

• The proposed weakly supervised clustering approach is effective for learning facial action units from large-scale, noisy web image data.
• This method offers a scalable and efficient alternative to fully supervised approaches for AU detection.
• The technique demonstrates potential for improving the robustness and accuracy of facial expression analysis systems.