3D real-time human reconstruction with a single RGBD camera.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• 3D human reconstruction is crucial for bridging the real and virtual worlds.
• Existing methods often require substantial computational resources, hindering real-time applications.
• There is a need for efficient and accessible 3D human reconstruction techniques.

Purpose of the Study:

• To develop a lightweight and efficient 3D human body reconstruction system.
• To enable real-time human modeling using only an RGBD camera.
• To improve the detail and accuracy of reconstructed human models.

Main Methods:

• A lightweight, end-to-end deep learning network, Fast Body Net (FBN), was developed.
• The FBN network prioritizes facial and hand details for enhanced local reconstruction.
• A denoising auto-encoder was trained to refine the human model's states.
• A new dataset, Indoor-Human, was created using Azure Kinect for training.

Main Results:

• The proposed system achieves at least 57% improvement in efficiency compared to state-of-the-art methods.
• The system maintains comparable accuracy while significantly boosting performance.
• Depth image features were utilized, eliminating the need for RGB data, contributing to FBN's lightweight design.

Conclusions:

• The developed lightweight system enables efficient real-time 3D human reconstruction.
• Consumer-grade RGBD cameras are suitable for real-time virtual reality display and interaction.
• The Fast Body Net and Indoor-Human dataset advance the field of accessible 3D human modeling.