MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

Area of Science:

• Computer Vision
• Machine Learning
• Human Pose Estimation

Background:

• Marker-based motion capture systems are effective but costly and restrictive.
• Recovering 3D human pose from single 2D images without markers presents significant challenges.
• Deep learning excels at 2D appearance feature extraction, but integrating this with 3D geometry recovery is complex.

Purpose of the Study:

• To develop a novel approach for accurate 3D full-body human pose recovery from single 2D images without markers.
• To integrate 2D appearance features with 3D geometry estimation, accounting for uncertainties.
• To enable markerless 3D human pose estimation for 'in-the-wild' images.

Main Methods:

• A deep fully convolutional neural network models uncertainty distributions for 2D joint locations (treated as latent variables).
• A sparse representation models unknown 3D poses.
• An Expectation-Maximization algorithm estimates 3D pose parameters, marginalizing out 2D uncertainties.

Main Results:

• The proposed approach achieves higher accuracy than state-of-the-art methods on benchmark datasets.
• Demonstrated successful application to 'in-the-wild' images using the MPII dataset.
• The method does not require synchronized 2D-3D training data.

Conclusions:

• The novel approach effectively recovers 3D human pose from single 2D images without markers.
• It offers a robust and accurate solution for markerless 3D human pose estimation.
• The method's applicability to unconstrained images opens new avenues for research and applications.