How much training data for facial action unit detection?

Area of Science:

• Computer Vision
• Machine Learning
• Biomedical Signal Processing

Background:

• Facial action unit (AU) detection is crucial for understanding human emotions and expressions.
• Optimizing training data size is essential for improving the accuracy and efficiency of AU detection models.

Purpose of the Study:

• To investigate how training set size, specifically the number of subjects and frames per subject, impacts appearance-based and shape-based facial AU detection.
• To determine the most efficient data sampling strategy for training AU detection classifiers.

Main Methods:

• Utilized digital video data from 80 subjects (over 350,000 frames) with expert-coded facial activity.
• Trained and tested support vector machine classifiers using shape-normalized SIFT descriptors (appearance features) and 66 facial landmarks (shape features).
• Employed ten-fold cross-validation to systematically vary the number of subjects and frames per subject.

Main Results:

• Appearance-based classifiers showed incremental performance improvement with increased subjects (8 to 64), irrespective of frames per subject (450-3600).
• Shape-based classifiers exhibited mixed results with varying numbers of subjects and frames.
• Maximal performance for appearance features was achieved with a large number of subjects and as few as 450 frames per subject.

Conclusions:

• The number of subjects in the training set is a more critical factor than the number of frames per subject for optimizing facial AU detection performance.
• Appearance-based features are more robust to variations in training data size, particularly benefiting from increased subject diversity.
• Findings suggest prioritizing subject diversity over data volume per subject for efficient and effective facial AU detection model training.