In the Pursuit of Effective Affective Computing: The Relationship Between Features and Registration

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Real-world facial expression recognition demands accurate tracking of unseen faces and movements in realistic settings.
• Dense facial point alignment (tracking 60-70 points) is ideal but historically lacked reliability and robustness.
• Current methods often use coarse alignment with appearance descriptors (e.g., HOG, Gabor) instead of dense alignment.

Purpose of the Study:

• To investigate the benefits of appearance-based representations versus standard pixel representations in facial expression recognition.
• To determine if appearance descriptors offer advantages beyond illumination invariance, particularly concerning alignment errors.
• To compare subject-dependent active appearance models (AAMs) with subject-independent constrained local models (CLMs) for dense alignment.

Main Methods:

• Comparison of dense alignment algorithms (AAMs vs. CLMs) on action-unit detection.
• Evaluation across multiple public datasets (CK+, Pain, M3, GEMEP-FERA).
• Assessment of performance under varying alignment accuracy and illumination conditions.

Main Results:

• Appearance descriptors provide no significant benefit over pixel representations when dense facial alignment is near-perfect and illumination is consistent.
• Appearance descriptors demonstrate robustness to alignment errors, outperforming pixel representations when misalignment occurs.
• Subject-independent CLMs showed competitive performance against subject-dependent AAMs in dense alignment tasks.

Conclusions:

• The utility of appearance descriptors in facial expression recognition is primarily linked to their robustness against alignment inaccuracies, not inherent advantages over pixel data under ideal conditions.
• Advancements in dense alignment algorithms like CLMs improve reliability, making high-accuracy facial tracking more feasible for real-world applications.
• Future research should focus on improving dense alignment robustness to further enhance facial expression recognition systems.