SERES: Semantic-Aware Neural Reconstruction From Sparse Views.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Neural Rendering

Background:

• Generating high-fidelity 3D models from sparse images is challenging due to radiance ambiguity.
• Existing methods struggle with feature mismatches in sparse input data.

Purpose of the Study:

• To propose a novel semantic-aware neural reconstruction method.
• To improve the accuracy and reduce ambiguity in 3D model generation from sparse inputs.

Main Methods:

• Enriching neural implicit representations with patch-based semantic logits.
• Optimizing signed distance fields and radiance fields concurrently.
• Introducing a novel regularization using geometric primitive masks to mitigate shape ambiguity.

Main Results:

• Achieved a 44% reduction in average chamfer distance for SparseNeuS and 20% for VolRecon on the DTU dataset.
• Reduced average error by 69% for NeuS and 68% for Neuralangelo when used as a plugin.

Conclusions:

• The proposed semantic-aware method effectively enhances 3D reconstruction from sparse images.
• Semantic information and geometric regularization are crucial for resolving ambiguity and improving fidelity.