CloCap-GS: Clothed Human Performance Capture With 3D Gaussian Splatting

Area of Science:

• Computer Vision
• Computer Graphics
• Human Performance Capture

Background:

• Previous 3D human and clothing reconstruction methods struggle with dynamic, arbitrary motions or have inefficient rendering.
• Existing techniques often fail to accurately track deforming geometries or synthesize high-quality novel views.

Purpose of the Study:

• To introduce CloCap-GS, a novel method for clothed human performance capture using 3D Gaussian Splatting.
• To address limitations in tracking stability and synthesis quality of prior approaches.

Main Methods:

• Aligning 3D Gaussians with deforming body and clothing geometries.
• Utilizing photometric constraints from video frames for temporal deformation recovery.
• Jointly optimizing geometry and Gaussian properties for dense tracking and novel-view synthesis.
• Incorporating a physics-aware, material-varying cloth model for plausible dynamics and interactions.

Main Results:

• Achieved improved accuracy in dense geometry tracking compared to existing methods.
• Enhanced the quality of novel-view synthesis for various garment types, including loose clothing.
• Demonstrated effectiveness on real-world sparse-view and monocular videos.

Conclusions:

• CloCap-GS offers a robust solution for clothed human performance capture from videos.
• The method successfully handles complex body and clothing dynamics, enabling high-fidelity 3D reconstruction and rendering.