LASOR: Learning Accurate 3D Human Pose and Shape via Synthetic Occlusion-Aware Data and Neural Mesh Rendering.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Human Pose and Shape Estimation

Background:

• Occlusion presents a significant challenge in human pose and shape estimation, particularly in real-world scenarios with inter-person, object-human, and self-occlusions.
• A scarcity of diverse and accurate training data exacerbates difficulties in handling occluded human figures.

Purpose of the Study:

• To develop a novel framework for estimating human pose and shape, specifically addressing inter-person, object-human, and self-occlusions.
• To enhance the accuracy and robustness of human pose and shape estimation in occluded environments.

Main Methods:

• A novel framework synthesizes occlusion-aware silhouette and 2D keypoint data for direct regression to SMPL (Skinned Multi-Person Linear Model) pose and shape parameters.
• A neural 3D mesh renderer is utilized for on-the-fly silhouette supervision, significantly improving shape estimation.
• Training data is synthesized using keypoints and silhouettes from panoramic viewpoints to address viewpoint diversity limitations in existing datasets.

Main Results:

• The proposed method achieves state-of-the-art performance on the 3DPW and 3DPW-Crowd datasets for pose estimation accuracy.
• The framework demonstrates superior performance in shape estimation compared to existing methods like Mesh Transformer, 3DCrowdNet, and ROMP.
• Top-tier accuracy in shape prediction is achieved on the SSP-3D dataset.

Conclusions:

• The developed framework effectively addresses occlusion challenges in human pose and shape estimation.
• The synthesis of occlusion-aware data and the use of neural rendering significantly advance the state-of-the-art in pose and shape estimation.
• The method offers a robust solution for accurately estimating human pose and shape in complex, real-world scenarios.