GANimation: Anatomically-aware Facial Animation from a Single Image.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Generative Adversarial Networks (GANs) have advanced facial expression synthesis.
• StarGAN, a successful architecture, conditions GANs using discrete expression domains.
• Existing methods are limited to predefined expressions based on dataset content.

Purpose of the Study:

• To develop a novel GAN conditioning scheme for continuous facial expression synthesis.
• To enable control over individual facial muscle movements (Action Units) and their combinations.
• To propose an unsupervised training strategy for enhanced robustness.

Main Methods:

• Introduced a GAN conditioning scheme based on Action Unit (AU) annotations.
• Utilized a continuous manifold to describe anatomical facial movements.
• Employed attention mechanisms for robustness against background and lighting variations.
• Developed a fully unsupervised training strategy requiring only AU-annotated images.

Main Results:

• Achieved synthesis of a wider range of expressions based on anatomically feasible muscle movements.
• Demonstrated superior performance compared to existing conditional generators.
• Showcased robustness in handling 'in the wild' images with varying conditions.
• Enabled fine-grained control over the magnitude and combination of AU activations.

Conclusions:

• The proposed AU-based GAN conditioning scheme significantly advances facial expression synthesis.
• This approach offers greater flexibility and control over expression generation.
• The unsupervised strategy and attention mechanisms enhance model applicability to real-world scenarios.