Free-HeadGAN: Neural Talking Head Synthesis With Explicit Gaze Control.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Computer Graphics

Background:

• Neural talking head synthesis aims to generate realistic animations of human faces.
• Existing methods often rely on complex 3D face models or lack precise control over subtle facial movements like eye gaze.

Purpose of the Study:

• To develop a person-generic neural talking head synthesis system (Free-HeadGAN) that achieves state-of-the-art performance.
• To demonstrate that sparse 3D facial landmarks are sufficient for high-quality generation, reducing reliance on complex priors.
• To enable explicit control over head pose, facial expressions, and eye gaze transfer.

Main Methods:

• Developed a pipeline with three core components: a 3D key-point estimator, a gaze estimation network, and a generator based on the HeadGAN architecture.
• Utilized sparse 3D facial landmarks for modeling, avoiding strong statistical priors like 3D Morphable Models.
• Incorporated an attention mechanism for few-shot learning capabilities when multiple source images are available.

Main Results:

• Achieved state-of-the-art generative performance in talking head synthesis.
• Demonstrated high photo-realism and superior identity preservation compared to existing reenactment and motion transfer methods.
• Successfully transferred eye gaze from a driving actor to a source identity, offering explicit gaze control.

Conclusions:

• Free-HeadGAN offers a robust and efficient approach to neural talking head synthesis.
• The method effectively models facial dynamics using sparse landmarks, enabling high-fidelity results and precise control.
• This work advances the field by providing a system with enhanced realism, identity preservation, and explicit gaze control.