Topo4D++: Realistic Physically Based 4D Head Capture With Topology-Preserving Gaussian Splatting and Expression...

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Reconstruction

Background:

• 4D head capture requires temporal correspondence for dynamic facial meshes.
• Existing methods lack accuracy and efficiency, relying on manual work or limited constraints.

Purpose of the Study:

• To propose Topo4D++, a novel framework for automatic 4D head geometry and texture reconstruction.
• To achieve a trade-off between accuracy and efficiency in dynamic 3D head modeling.

Main Methods:

• Representing facial models using dynamic 3D Gaussians with fixed topology, bound to mesh vertices.
• Leveraging 3D Gaussian Splatting (3DGS) for inverse rendering and tracking all vertices.
• Alternating optimization of geometry and texture with physical/topological constraints and blendshape priors.
• Training a diffusion model for BRDF texture map generation for physically based rendering.

Main Results:

• Successfully reconstructs densely aligned 4D heads and 8K BRDF maps from multi-view videos.
• Extracts dynamic facial meshes with regular wiring and high-fidelity, pore-level detailed textures.
• Demonstrates generalization across different capture systems, identities, and expressions.
• Outperforms state-of-the-art methods in both mesh and texture reconstruction.

Conclusions:

• Topo4D++ offers an efficient and accurate solution for automatic 4D head capture.
• The proposed dynamic 3D Gaussian representation and optimization strategy are effective for high-fidelity reconstruction.
• The JHead benchmark facilitates comprehensive evaluation of 4D head capture techniques.