Learning Representations for Facial Actions From Unlabeled Videos

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Manual labeling of facial action units (AUs) is expert-dependent, costly, and time-consuming.
• Facial action analysis often relies on supervised methods that require extensive labeled data.
• Unlabeled video data offers a vast, underutilized resource for training facial action recognition models.

Purpose of the Study:

• To develop a novel method for learning discriminative facial action representations from unlabeled videos.
• To overcome the challenge of disentangling facial actions from head movements in video data.
• To reduce the reliance on expert-annotated data for facial action unit detection.

Main Methods:

• Proposing a twin-cycle autoencoder (TAE) model to learn representations from pixel-wise displacements between sequential facial images.
• Training TAE to reconstruct target images while disentangling facial action-induced and head pose-induced movements.
• Evaluating the learned representations using action unit detection tasks and facial image retrieval.

Main Results:

• TAE achieves AU detection accuracy comparable to existing supervised methods.
• The model successfully learns to decouple facial action representations from head pose variations.
• Qualitative and quantitative analyses validate TAE's effectiveness in disentangling movement types.

Conclusions:

• The proposed twin-cycle autoencoder (TAE) effectively learns discriminative facial action representations from unlabeled video data.
• TAE offers a viable, cost-effective alternative to supervised methods by leveraging unlabeled videos.
• The method demonstrates robust performance in disentangling complex facial movements, paving the way for more efficient facial action analysis.