MulayCap: Multi-Layer Human Performance Capture Using a Monocular Video Camera.

Area of Science:

• Computer Vision
• Computer Graphics
• Human Performance Capture

Background:

• Accurate human performance capture is crucial for various applications.
• Existing single-view methods often require tedious 3D scanning for template mesh generation.
• Dynamic clothing reconstruction from monocular video remains a challenge.

Purpose of the Study:

• Introduce MulayCap, a novel monocular human performance capture method.
• Enable realistic reconstruction of dynamic clothed humans without pre-scanning.
• Facilitate advanced editing applications through semantic modeling.

Main Methods:

• Employ a multi-layer representation for geometry and texture.
• Utilize a Garment-from-Video (GfV) method with cloth simulation and gradient descent for dynamic garment reconstruction.
• Decompose images into shading and albedo layers for texture rendering, fusing a fixed albedo map.

Main Results:

• Achieved realistic rendering of dynamically changing details from monocular video.
• Bypassed the need for time-consuming 3D scanning of human-specific mesh templates.
• Demonstrated superior performance compared to existing single-view human performance capture systems.

Conclusions:

• MulayCap offers a robust and efficient solution for monocular human performance capture.
• The multi-layer approach enables high-fidelity reconstruction of clothed humans.
• The method supports diverse editing applications including cloth editing, relighting, and AR.