Implicit Neural Representations With Structured Latent Codes for Human Body Modeling.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Implicit neural representations excel at novel view synthesis with dense camera views.
• Learning 3D representations becomes ill-posed with sparse input views.
• Existing methods struggle with limited camera perspectives for human performance capture.

Purpose of the Study:

• To develop a robust method for novel view synthesis and 3D reconstruction from sparse camera views.
• To introduce a new human body representation that effectively integrates information across video frames.
• To address the ill-posed nature of learning 3D representations from limited data.

Main Methods:

• Proposed Neural Body, a novel human body representation leveraging shared latent codes anchored to a deformable mesh.
• Integrated observations across video frames to overcome sparsity limitations.
• Combined Neural Body with implicit surface models for enhanced geometric accuracy.

Main Results:

• Achieved superior novel view synthesis and 3D reconstruction performance compared to prior works on both synthetic and real-world data.
• Demonstrated effective reconstruction of moving humans from monocular video.
• Validated the efficiency of learning 3D representations with geometric guidance from the deformable mesh.

Conclusions:

• Neural Body successfully addresses the challenge of 3D human performance capture from sparse views by integrating temporal information.
• The proposed representation enables high-fidelity novel view synthesis and accurate 3D reconstruction.
• This method offers a significant advancement for applications requiring human body modeling from limited sensor data.