Single-Shot 3D Multi-Person Shape Reconstruction from a Single RGB Image

Area of Science:

• Computer Vision
• Machine Learning
• 3D Computer Graphics

Background:

• Current 3D human shape reconstruction methods often focus on single individuals.
• Existing techniques typically require ground-truth depth information for camera coordinate system conversion.
• Root-relative 3D shape reconstruction is common, lacking absolute positional data.

Purpose of the Study:

• To develop an end-to-end learning-based model for single-shot, 3D, multi-person shape reconstruction.
• To achieve reconstruction directly in the camera coordinate system from a single RGB image.
• To enable real-time processing of 3D multi-person shape reconstruction.

Main Methods:

• A novel end-to-end deep learning network is proposed for single-shot 3D multi-person shape reconstruction.
• The network utilizes output tensors divided into grid cells, with each cell containing subject-specific information.
• The model predicts the absolute root joint position alongside root-relative 3D shape reconstruction.

Main Results:

• The proposed network enables single-shot 3D reconstruction of multiple persons in the camera coordinate system.
• The model successfully reconstructs root-relative 3D shapes and predicts absolute root joint positions.
• The system achieves real-time performance, processing images at approximately 37 frames per second.

Conclusions:

• The developed model offers a significant advancement in 3D multi-person shape reconstruction from single RGB images.
• This approach overcomes limitations of previous methods by handling multiple subjects and directly outputting results in the camera coordinate system.
• The real-time capability makes the method suitable for various practical applications.