VMarker-Pro: Probabilistic 3D Human Mesh Estimation From Virtual Markers.

Area of Science:

• Computer Vision
• 3D Human Pose and Shape Estimation
• Machine Learning

Background:

• Monocular 3D human mesh estimation is challenged by depth ambiguity and mapping complexity.
• Existing methods using 3D poses lose vital body shape information, while motion capture systems are impractical for real-world images.
• There is a need for robust and accurate methods for markerless 3D human shape reconstruction from single images.

Purpose of the Study:

• To introduce an innovative intermediate representation using virtual markers for improved 3D human mesh estimation.
• To develop VMarker for detecting virtual markers and reconstructing intact human meshes from wild images.
• To enhance robustness in occluded scenarios with VMarker-Pro, a probabilistic framework utilizing diffusion models.

Main Methods:

• Learned virtual markers from large-scale motion capture data to mimic physical markers.
• Developed VMarker to detect virtual markers from images and interpolate to obtain 3D human meshes.
• Proposed VMarker-Pro, a probabilistic framework with diffusion models to handle occlusions by generating multiple plausible meshes.

Main Results:

• The proposed VMarker and VMarker-Pro methods surpass existing approaches on three benchmark datasets.
• Significant improvements were observed on the SURREAL dataset, known for diverse body shapes.
• VMarker-Pro demonstrated superior performance in occluded scenarios by accurately modeling data distributions.

Conclusions:

• Virtual markers offer a promising intermediate representation for monocular 3D human mesh estimation.
• VMarker and VMarker-Pro provide accurate, robust, and shape-aware 3D human reconstruction from images.
• The probabilistic framework effectively addresses occlusion challenges, advancing the state-of-the-art in markerless 3D human shape estimation.