Machine and deep learning in facial expression recognition: a survey based on facial action units

Area of Science:

• Computer Science, Artificial Intelligence, Machine Learning, Deep Learning, Human-Computer Interaction, Affective Computing.

Background:

• Facial Expression Recognition (FER) is crucial for autonomous systems in security, healthcare, and human-robot interaction.
• Accurate interpretation of human emotions via facial cues enables intelligent systems to understand user intent in real-time.
• Recent advancements focus on machine learning (ML) and deep learning (DL) for enhanced FER capabilities.

Purpose of the Study:

• To provide a comprehensive survey of recent FER methodologies, emphasizing ML and DL approaches from 2019 to 2024.
• To analyze the role of Facial Action Unit Detection (FAUD) as an intermediate representation for improved FER.
• To identify trends, performance patterns, and the advantages of FAUD-based modeling in FER.

Main Methods:

• Systematic literature review of FER and FAUD research published between 2019 and 2024.
• Categorization of analyzed studies based on their approach: FAUD-oriented, FER-oriented, AU-assisted FER, and FAUD-to-FER.
• Structured comparison of methodologies, performance metrics, and application contexts.

Main Results:

• Identification of key methodological trends in ML and DL for FER and FAUD.
• Analysis of reported performance patterns across different approaches.
• Evidence suggesting that FAUD-based intermediate modeling offers significant advantages for reliable FER in specific scenarios.

Conclusions:

• The integration of FAUD as an intermediate step enhances the reliability and interpretability of FER systems.
• ML and DL continue to drive progress in FER, with FAUD-based methods showing particular promise.
• Future research should further explore the synergistic potential of FAUD and advanced DL techniques for robust emotion recognition.