MPS-NeRF: Generalizable 3D Human Rendering From Multiview Images.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computer Graphics

Background:

• Neural Radiance Fields (NeRF) have advanced 3D human rendering, including novel view synthesis and pose animation.
• Existing NeRF methods typically require person-specific training and multi-view videos, limiting generalization to unseen subjects.

Purpose of the Study:

• To address the challenge of rendering novel views and poses for individuals not encountered during training, using only multi-view still images.
• To develop a generalizable NeRF model capable of handling unseen subjects from static image inputs.

Main Methods:

• Proposed a method to train a generalizable NeRF using multi-view images as conditional input.
• Introduced a dedicated representation combining a canonical NeRF with a volume deformation scheme.
• Leveraged a parametric 3D human model fitted to input images to derive volume deformation for connecting canonical and image spaces.

Main Results:

• The proposed method successfully renders novel views and novel poses for unseen individuals.
• Experiments on real and synthetic data demonstrated the efficacy of the approach for both novel view synthesis and pose animation tasks.
• The canonical space and volume deformation effectively enabled generalization to different people.

Conclusions:

• The developed method offers a significant advancement in 3D human rendering from limited input data.
• The approach shows strong potential for applications requiring flexible and generalizable human rendering without extensive training data.
• This work paves the way for more robust and widely applicable NeRF-based human modeling techniques.