F³A-GAN: Facial Flow for Face Animation With Generative Adversarial Networks

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Face animation synthesizes continuous face images from a source image using conditional motion.
• Existing 1D/2D motion representations (e.g., action units, landmarks) yield suboptimal results in complex scenarios like continuous generation and large pose changes.

Purpose of the Study:

• To develop a novel representation for face motion that preserves motion information and ensures geometric continuity.
• To create a synthesis framework for high-quality, continuous face animation using the proposed representation.

Main Methods:

• Proposed a novel 3D geometric flow representation, termed facial flow, to model natural human face motion.
• Developed a hierarchical conditional framework that integrates multi-scale image appearance features with facial flow motion features.
• Designed a progressive decoding process to generate continuous face images from fused features.

Main Results:

• Facial flow effectively controls continuous changes in the face, outperforming previous 1D/2D representations.
• The hierarchical framework successfully combines appearance and motion features for improved synthesis.
• Experimental results demonstrate superior performance compared to state-of-the-art face animation methods.

Conclusions:

• The proposed facial flow representation and synthesis framework significantly enhance the quality and realism of continuous face animation.
• This approach addresses limitations of previous methods, particularly in handling complex motion and pose transformations.