Neural light field representation and reconstruction based on a ray displacement field

Area of Science:

• Computer Vision
• Computer Graphics
• Computational Imaging

Background:

• High-quality light field (LF) acquisition faces a spatial-angular resolution trade-off.
• Hybrid camera systems improve acquisition but pose reconstruction challenges due to sparse sampling and occlusions.

Purpose of the Study:

• To propose a geometry-aware implicit neural representation (INR) framework for robust light field reconstruction.
• To address challenges in dense reconstruction from sparse, multi-resolution hybrid camera data.

Main Methods:

• Developed a compact, continuous representation combining a radiance field and a geometry module.
• Utilized scalar disparity for geometric interpretation and a ray-displacement field for occlusion-aware reconstruction.
• Leveraged global epipolar geometry and local displacement correction for geometric consistency.

Main Results:

• Demonstrated end-to-end differentiable optimization from sparse inputs without external training data.
• Achieved high-frequency detail preservation and geometric consistency in experiments.
• Successfully applied to hybrid camera data fusion and spatial-angular super-resolution.

Conclusions:

• The proposed geometry-aware INR framework offers a powerful solution for light field reconstruction.
• The method effectively handles occlusions and non-Lambertian effects, improving reconstruction quality.
• Enables high-fidelity LF reconstruction from challenging sparse and multi-resolution data.