Learning Reliable Gradients From Undersampled Circular Light Field for 3D Reconstruction

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computational Imaging

Background:

• Undersampled circular light fields (LF) pose challenges for accurate 3D reconstruction due to broken epipolar trajectories.
• Existing methods struggle with degraded accuracy caused by discrete segments in the circular epipolar plane volume (CEPV).

Purpose of the Study:

• To develop a robust 3D reconstruction algorithm for undersampled circular LFs.
• To enhance 3D reconstruction accuracy by leveraging the inherent coherent structures within discrete CEPV trajectories.

Main Methods:

• A mask-guided Convolutional Neural Network (CNN) combined with a Long Short-Term Memory (LSTM) network was proposed.
• The network learns to map low-angular-sampling CEPV data to high-angular-sampling gradients.
• A reliable-mask-based loss function was incorporated to segment accurate gradient regions.

Main Results:

• The proposed method successfully reconstructs scene points using gradients from reliable EPI regions.
• Experimental results on synthetic and real-world datasets demonstrate superior performance compared to state-of-the-art methods.
• Accurate depth and foreground/background segmentation were achieved.

Conclusions:

• The developed algorithm effectively overcomes the limitations of undersampled circular LFs for 3D reconstruction.
• The mask-guided CNN+LSTM approach provides a significant improvement in 3D reconstruction accuracy.
• This work offers a promising solution for high-quality 3D scene recovery from sparse light field data.