HP-Gaussian: Head Prior-Guided Gaussian Splatting for Personalized Talking Head Synthesis From Few-Second Video

Area of Science:

• Computer Vision
• Artificial Intelligence
• Graphics

Background:

• Gaussian Splatting (GS) advances talking head synthesis but lacks generalization for new identities.
• Existing GS methods often require extensive identity-specific training data.

Purpose of the Study:

• To develop a novel method, Head Prior guided Gaussian Splatting (HP-Gaussian), for personalized talking head synthesis.
• To achieve generalization to new identities using only a few training examples.

Main Methods:

• HP-Gaussian directly predicts Gaussian parameters from multi-modal inputs (audio, visual cues) using a feed-forward design.
• A Spatial Gaussian Transformer enhances Gaussian feature learning by capturing inter-Gaussian correlations.
• A two-stage training strategy involves pre-training on diverse identities and then personalized adaptation using short videos.

Main Results:

• HP-Gaussian demonstrates effective generalization to new identities with limited training data.
• The method synthesizes high-fidelity and personalized talking videos.
• Achieves a new benchmark in efficiency and quality for talking head synthesis.

Conclusions:

• HP-Gaussian offers a significant improvement over existing methods for personalized talking head synthesis.
• The approach enables efficient and high-quality generation of customized talking videos.
• The proposed method sets a new standard for few-shot learning in this domain.