Instant Gaussian Splatting Generation for High-Quality and Real-Time Facial Asset Rendering

Area of Science:

• Computer Graphics
• Artificial Intelligence
• 3D Modeling

Background:

• Traditional and AI methods for 3D asset generation face quality-speed trade-offs in editing and rendering.
• Efficient rendering of high-fidelity facial meshes with textures remains a challenge.

Purpose of the Study:

• To propose GauFace, a Gaussian Splatting representation for efficient facial mesh rendering.
• To introduce TransGS, a diffusion transformer for rapid generation of GauFace assets.
• To enable real-time facial interaction and editing capabilities.

Main Methods:

• Developed GauFace, a novel Gaussian Splatting representation for facial assets.
• Introduced TransGS, a diffusion transformer utilizing a patch-based pipeline and texel-aligned sampling with UV positional encoding.
• Trained TransGS for instant generation of GauFace assets from mesh, textures, and lighting.

Main Results:

• TransGS generates GauFace assets in 5 seconds.
• Achieved real-time facial interaction at 30fps@1440p on a mobile platform (Snapdragon 8 Gen 2).
• Demonstrated superior performance compared to traditional renderers and recent neural rendering methods in evaluations and user studies.

Conclusions:

• GauFace and TransGS offer a significant advancement in efficient and high-fidelity facial asset rendering.
• The approach enables versatile applications across various platforms, including PCs, phones, and VR headsets.
• Facilitates editing and animation comparable to traditional CG pipelines.