Training Affective Computer Vision Models by Crowdsourcing Soft-Target Labels.

Area of Science:

• Computer Science
• Psychology
• Human-Computer Interaction

Background:

• Traditional emotion detection classifiers predict discrete emotions, but human expressions are subjective and ambiguous.
• Handling compound and ambiguous emotion labels requires methods that account for diverse human interpretations.

Purpose of the Study:

• To explore the feasibility of using crowdsourcing for acquiring reliable soft-target emotion labels.
• To evaluate an emotion detection classifier trained with crowdsourced soft-target labels.
• To determine if crowdsourcing can generate emotion label distributions mirroring lab settings.

Main Methods:

• Utilized the Child Affective Facial Expression (CAFE) dataset with 207 images.
• Acquired crowd labels via Microworkers, evaluating both unfiltered and filtered workers.
• Trained two ResNet-152 classifiers on soft-target labels: one with one-hot encoding, another with distribution vectors.
• Compared classifier outputs to human label distributions using L1 distances and t-tests.

Main Results:

• Filtered crowd workers showed high agreement with CAFE labels (100% for happy, neutral, sad, fear+surprise; 88.8% for anger+disgust).
• While one-hot encoded classifiers had higher F1-scores (94.33% vs. 78.68%), the crowd-trained classifier's output distribution better matched human label distributions (t=3.2827, p=0.0014).

Conclusions:

• Crowdsourcing, with effective worker filtering, is a viable method for obtaining soft-target emotion labels.
• Emotion probability distributions reflecting subjective human interpretation are valuable for affective computing applications.
• This approach enhances emotion detection by representing the diversity of human perception.