Facial Action Unit Recognition and Intensity Estimation Enhanced through Label Dependencies

Area of Science:

• Computer Vision
• Machine Learning
• Biomedical Engineering

Background:

• Facial action unit (AU) dependencies, crucial for accurate recognition and intensity estimation, are often underutilized.
• Existing methods struggle to fully exploit the inherent anatomic and correlational relationships between AUs.

Purpose of the Study:

• To propose novel hybrid Bayesian network methods for recognizing AUs and estimating their intensity.
• To effectively model and leverage complex, global inherent AU dependencies.
• To enhance AU recognition and intensity estimation using facial expression information.

Main Methods:

• Developed hybrid Bayesian networks combining latent regression Bayesian networks (LRBNs) and Bayesian networks (BNs).
• Utilized LRBNs for modeling relationships between multiple AUs or intensities, with visible nodes representing ground-truth data.
• Incorporated facial image measurements and AU dependencies in lower BN layers for recognition and intensity estimation.
• Proposed efficient learning algorithms for the hybrid models.
• Extended models for facial expression-assisted AU recognition and intensity estimation.

Main Results:

• Demonstrated that the proposed hybrid Bayesian networks effectively model complex inherent AU dependencies.
• Achieved accurate AU recognition and intensity estimation by exploiting AU relationships.
• Showcased that incorporating expression labels during training significantly boosts the estimation of AU dependencies.

Conclusions:

• The novel hybrid Bayesian network approach successfully models inherent AU dependencies for improved recognition and intensity estimation.
• Facial expression information serves as a valuable addition to enhance the performance of AU analysis.
• The proposed methods offer a significant advancement in the field of automated facial behavior analysis.