NOPE-SAC: Neural One-Plane RANSAC for Sparse-View Planar 3D Reconstruction

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Sparse-view 3D reconstruction is challenging due to insufficient correspondences for accurate camera pose estimation.
• Existing methods struggle with severe viewpoint changes and limited input data.

Purpose of the Study:

• To develop a robust framework for accurate camera pose estimation in sparse-view 3D reconstruction.
• To address the limitations of insufficient correspondences in two-view reconstruction.

Main Methods:

• Introduced the Neural One-PlanE RANSAC (NOPE-SAC) framework.
• Utilized a Siamese network for plane detection and learned 3D plane correspondences.
• Employed shared MLPs for estimating one-plane camera pose hypotheses, refined using RANSAC.

Main Results:

• NOPE-SAC effectively handles sparse-view inputs with severe viewpoint changes.
• Achieved significant improvements in camera pose estimation accuracy.
• Set new state-of-the-art performances on MatterPort3D and ScanNet benchmarks.

Conclusions:

• The NOPE-SAC framework enables stable pose voting and reliable refinement with minimal plane correspondences.
• Demonstrates superior performance for challenging sparse-view 3D reconstruction tasks.