SN$^{2}$2eRF: A Framework for Neural Radiance Fields Given Sparse and Noisy Poses.

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Reconstruction

Background:

• Neural Radiance Fields (NeRFs) excel at photorealistic novel view synthesis.
• Training NeRFs typically requires hundreds of precisely posed input views, limiting their use in real-world scenarios like indoor rooms.

Purpose of the Study:

• To develop a framework for reconstructing Neural Radiance Fields (NeRFs) using significantly fewer and less accurate input views.
• To enable efficient and robust NeRF training for room-scale scenes despite pose inaccuracies and sparse data.

Main Methods:

• Propose SN ²eRF, a novel framework integrating multi-view and monocular priors.
• Utilize key point extraction and matching to constrain pose optimization.
• Employ Ray Transformer with a monocular depth estimator for dense depth prior geometry optimization.

Main Results:

• Achieve state-of-the-art accuracy in novel view synthesis for indoor environments.
• Demonstrate successful NeRF reconstruction with substantially reduced view count and improved tolerance to noisy poses.
• Validate the effectiveness of leveraging combined multi-view and monocular information.

Conclusions:

• SN ²eRF significantly overcomes the data requirements of traditional NeRF training.
• The framework offers a practical solution for reconstructing complex indoor scenes with limited and imprecise input data.
• This advancement broadens the applicability of NeRFs in real-world 3D scene reconstruction tasks.