3D Talking Face With Personalized Pose Dynamics.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Current 3D talking face generation methods often produce static head poses, which is unrealistic.
• Existing approaches for dynamic head pose generation typically overlook personality attributes.
• There is a need for methods that integrate personalized head movements with realistic 3D face generation.

Purpose of the Study:

• To propose a unified audio-driven framework for generating 3D talking faces with personalized pose dynamics.
• To address the limitations of static head poses in current 3D face generation techniques.
• To incorporate personality-specific head movements into 3D facial animation.

Main Methods:

• Developed a novel person-specific dataset with corresponding head poses and face shapes.
• Proposed a two-module framework: PoseGAN for head pose sequence generation and PGFace for natural face model generation.
• Utilized audio input to drive both head pose dynamics and facial expression generation.

Main Results:

• Successfully generated person-specific head pose sequences synchronized with input audio.
• The proposed method creates 3D talking heads with dynamic head movements that align with human perception.
• Experimental results validate the effectiveness of the unified approach in producing natural and personalized 3D facial animations.

Conclusions:

• The developed audio-driven approach effectively generates 3D talking faces with personalized and dynamic head poses.
• This method enhances the realism and human-like quality of 3D facial animations by incorporating personality.
• The framework offers a significant advancement in creating more engaging and lifelike 3D digital characters.