FER20E: An Extended Facial Expression Recognition Dataset with 20 Discrete Emotions

Area of Science:

• Computer Vision
• Affective Computing
• Human-Computer Interaction

Background:

• Traditional Facial Emotion Recognition (FER) systems are limited to basic emotions, failing to capture real-world emotional complexity and cultural nuances.
• Existing datasets often lack the granularity and psychological validity required for advanced FER applications.

Purpose of the Study:

• Introduce FER20E, a large-scale dataset with 20 emotion categories, including fine-grained affective states.
• Develop a robust emotion taxonomy integrating Facial Action Coding System (FACS) Action Units (AUs) and the valence-arousal model.
• Provide a comprehensive benchmark for advancing FER in unconstrained, real-world scenarios.

Main Methods:

• Created FER20E, a dataset with 20 emotion categories derived from FACS AUs and the valence-arousal model.
• Developed a semi-automated, human-in-the-loop data annotation tool (DL-DAT) for scalable and reliable annotation.
• Evaluated state-of-the-art models (CNNs, Transformers) on the FER20E dataset.

Main Results:

• Lightweight models (MobileNetV2, SqueezeNet) achieved competitive performance for real-time FER.
• Transformer-based models (ViT21k) demonstrated superior accuracy, emphasizing the value of large-scale pretraining.
• Identified challenges in emotion ambiguity, AU overlap, and cross-cultural variations.

Conclusions:

• The FER20E dataset offers a valuable resource for developing more sophisticated and culturally aware FER systems.
• Fine-grained FER is crucial for addressing the complexities of human emotion in real-world applications.
• The dataset and tools facilitate research in advanced emotion recognition, supporting both efficiency and accuracy.