Learning 3D Human Shape and Pose From Dense Body Parts.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Human Pose Estimation

Background:

• Reconstructing 3D human shape and pose from monocular images presents significant challenges due to non-linear image-to-model mappings and joint position drift in rotation-based pose representations.
• Existing learning-based methods show promise but often struggle with misalignment issues.

Purpose of the Study:

• To develop a novel deep learning approach for accurate 3D human shape and pose reconstruction from single images.
• To address the limitations of current methods by leveraging dense correspondences and improved pose representation.

Main Methods:

• Proposes the Decompose-and-aggregate Network (DaNet) utilizing dense correspondence maps as intermediate representations for 2D-to-3D mapping.
• Employs decomposed global and local prediction streams for comprehensive shape and pose perception.
• Introduces a position-aided rotation feature refinement and a Part-based Dropout (PartDrop) strategy for robust pose prediction and improved feature learning.

Main Results:

• DaNet effectively bridges 2D pixel information to 3D vertices through dense correspondences.
• The decomposed network architecture and aggregation strategy enhance both global and fine-grained shape and pose predictions.
• PartDrop encourages focus on complementary body parts and spatial relationships, leading to more robust results.

Conclusions:

• The proposed DaNet method significantly improves 3D human shape and pose reconstruction performance compared to state-of-the-art approaches.
• The use of dense correspondences and the novel network architecture effectively mitigate common misalignment issues.
• The method demonstrates strong efficacy across diverse indoor and real-world datasets, including Human3.6M, UP3D, COCO, and 3DPW.