A Deeper Look into DeepCap (Invited Paper).

Area of Science:

• Computer Vision
• Machine Learning
• 3D Reconstruction

Background:

• Human performance capture is crucial for applications like film and VR.
• Existing methods often need multi-view systems or lack dense, coherent geometry.
• Frame-to-frame correspondences are challenging to achieve.

Purpose of the Study:

• To develop a novel deep learning approach for monocular dense human performance capture.
• To eliminate the need for 3D ground truth annotations during training.
• To improve the quality and robustness of human performance capture.

Main Methods:

• A weakly supervised training strategy using multi-view supervision.
• A network architecture with two separate networks for pose estimation and non-rigid surface deformation.
• Monocular input for performance capture.

Main Results:

• The proposed method outperforms existing state-of-the-art approaches.
• Achieves high quality and robustness in dense space-time coherent geometry recovery.
• Demonstrates effective performance without 3D ground truth.

Conclusions:

• The novel deep learning approach enables efficient and accurate monocular dense human performance capture.
• Weakly supervised training significantly reduces data annotation requirements.
• The method offers a robust solution for various computer vision applications.