StructNeRF: Neural Radiance Fields for Indoor Scenes With Structural Hints.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Neural Radiance Fields (NeRF) excel at photorealistic view synthesis with dense input images.
• Sparse input views severely under-constrain NeRF geometry, degrading novel view synthesis quality.
• Existing methods struggle with unconstrained geometry in sparse-view NeRF.

Purpose of the Study:

• To propose StructNeRF, a novel solution for robust novel view synthesis from sparse inputs in indoor scenes.
• To address the under-constrained geometry problem in NeRF when using limited input views.
• To improve both geometric accuracy and rendering quality in sparse-view NeRF.

Main Methods:

• StructNeRF utilizes structural hints from multi-view inputs to constrain NeRF geometry.
• A patch-based multi-view consistent photometric loss is applied to textured regions.
• Non-textured regions are explicitly constrained to be 3D consistent planes.
• Dense self-supervised depth constraints are integrated to guide the optimization process.

Main Results:

• StructNeRF significantly improves the geometric consistency and view synthesis performance of NeRF with sparse inputs.
• The method demonstrates superior or comparable results against state-of-the-art methods like NeRF, DSNeRF, RegNeRF, Dense Depth Priors, and MonoSDF.
• Quantitative and qualitative evaluations on real-world datasets validate the effectiveness of StructNeRF.

Conclusions:

• StructNeRF effectively handles the under-constrained geometry problem in sparse-view NeRF by incorporating structural priors.
• The proposed method offers a robust and efficient approach for novel view synthesis in challenging sparse-input scenarios.
• StructNeRF advances the capabilities of NeRF for practical applications requiring high-quality rendering from limited data.