Facial Action Unit Detection via Adaptive Attention and Relation

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Facial Action Unit (AU) detection is crucial for understanding human emotions and expressions.
• Existing methods struggle with subtle, dynamic AUs and often oversimplify or miss crucial relational information.
• Current approaches may discard essential facial regions or include irrelevant areas in attention mechanisms.

Purpose of the Study:

• To develop a novel Adaptive Attention and Relation (AAR) framework for improved facial AU detection.
• To address limitations in capturing correlated information from subtle and dynamic AUs.
• To enhance the precision of regional correlation distribution learning for each AU.

Main Methods:

• Proposed an adaptive attention regression network to learn global attention maps for each AU, balancing landmark-based and global dependencies.
• Developed an adaptive spatio-temporal graph convolutional network for reasoning independent, inter-dependent, and temporal AU patterns.
• Integrated these components into a unified AAR framework for facial AU detection.

Main Results:

• Achieved competitive performance on challenging benchmarks: BP4D, DISFA, GFT (constrained), and Aff-Wild2 (unconstrained).
• Demonstrated precise learning of regional correlation distributions for individual AUs.
• The AAR framework effectively captures both specified and globally distributed facial dependencies.

Conclusions:

• The proposed AAR framework offers a significant advancement in facial AU detection.
• The adaptive attention and spatio-temporal reasoning mechanisms are key to handling AU complexity.
• This approach shows strong potential for real-world applications in emotion recognition and human-computer interaction.