HDhuman: High-Quality Human Novel-View Rendering From Sparse Views.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Novel view synthesis of human performers with complex clothing textures from sparse views is challenging.
• Existing methods struggle to recover high-frequency geometric details from complex textures, limiting rendering quality.

Purpose of the Study:

• To propose HDhuman, a novel framework for high-quality human reconstruction and rendering from sparse views.
• To address the limitations of current neural rendering techniques in handling intricate clothing patterns.

Main Methods:

• Utilizes a human reconstruction network with a pixel-aligned spatial transformer for detailed geometry recovery.
• Employs a rendering network with geometry-guided pixel-wise feature integration for enhanced multi-view feature fusion.
• The pixel-aligned spatial transformer identifies correlations across input views to reconstruct high-frequency details.

Main Results:

• Achieves high-quality human reconstruction and rendering at 2k resolution on novel views.
• Demonstrates superior performance compared to existing generic and specific methods on both synthetic and real-world datasets.
• The proposed method generalizes to novel subjects without requiring scene-specific training or fine-tuning.

Conclusions:

• HDhuman effectively reconstructs and renders humans with complex textures from sparse views.
• The framework offers a generalizable solution for neural rendering, outperforming prior approaches.
• Publicly available code and data facilitate further research in human view synthesis.