ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Current audio-driven facial animation often lacks emotional depth and style.
• Generating high-quality talking head videos with integrated emotion and style is challenging.

Purpose of the Study:

• To propose ESGaussianFace, an innovative framework for emotional and stylized audio-driven facial animation.
• To improve the generation of 3D consistent, high-quality talking head videos.

Main Methods:

• Leveraging 3D Gaussian Splatting for scene reconstruction and video rendering.
• Implementing an emotion-audio-guided spatial attention mechanism for feature integration.
• Utilizing two 3D Gaussian deformation predictors for emotion and style-driven transformations.
• Adopting a multi-stage training strategy for sequential learning of facial dynamics.

Main Results:

• ESGaussianFace achieves high efficiency, quality, and 3D consistency in generated videos.
• The method demonstrates superior performance in lip movement accuracy, expression variation, and style expressiveness compared to state-of-the-art techniques.

Conclusions:

• ESGaussianFace effectively integrates emotional and style features into audio-driven facial animation.
• The proposed framework advances the state-of-the-art in generating realistic and expressive talking head videos.