UncNeRF: Uncovering Heavily Occluded Object With Multi-View Clues

Area of Science:

• Computer Vision
• 3D Reconstruction
• Neural Rendering

Background:

• Neural Radiance Fields (NeRF) excel at photorealistic rendering but struggle with occluded objects.
• Existing methods often rely on external 2D priors, lacking 3D consistency for occluded object reconstruction.

Purpose of the Study:

• To develop a method, UncNeRF, for reconstructing heavily occluded objects using multi-view defective images.
• To introduce novel techniques for enhancing 3D object reconstruction in the presence of significant occlusion.

Main Methods:

• Utilizes multi-view clues from captured defective images to uncover occluded objects.
• Employs object-centric forward warping for multi-view optimization.
• Enhances reconstruction via pseudo-training views and spatial-relation regularization.

Main Results:

• Achieves state-of-the-art performance in heavy occlusion removal.
• Demonstrates superior reconstruction of occluded objects compared to existing methods.
• Introduces the Heavy Occlusion Removal (HOR) dataset for evaluating occluded object reconstruction.

Conclusions:

• UncNeRF effectively addresses the challenge of reconstructing heavily occluded objects.
• The proposed method offers a robust solution for 3D object recovery in practical, occluded scenarios.
• The HOR dataset provides a valuable benchmark for future research in occlusion removal.