Geo-NI: Geometry-Aware Neural Interpolation for Light Field Rendering.

Area of Science:

• Computer Vision
• Computer Graphics
• Image Processing

Background:

• Existing learning-based light field rendering methods include Neural Interpolation (NI) and Depth Image-Based Rendering (DIBR).
• NI excels at direct interpolation but struggles with non-Lambertian effects and large disparities.
• DIBR leverages scene geometry but can be less effective with ambiguous depth information.

Purpose of the Study:

• To develop a novel framework, Geometry-aware Neural Interpolation (Geo-NI), for light field rendering.
• To combine the strengths of NI and DIBR to overcome their individual limitations.
• To enhance novel view synthesis quality, particularly for challenging scenarios like large disparities and non-Lambertian effects.

Main Methods:

• Proposed Geo-NI framework integrates Neural Interpolation within a Depth Image-Based Rendering pipeline.
• A DIBR network constructs a novel reconstruction cost volume for neural interpolated light fields based on depth hypotheses.
• An efficient modeling strategy is proposed to encode high-dimensional cost volumes using a lower-dimension network.

Main Results:

• The Geo-NI framework successfully renders views with large disparities by utilizing scene geometry.
• It effectively reconstructs non-Lambertian effects even when depth information is ambiguous.
• Extensive experiments demonstrate superior performance compared to existing methods across various datasets.

Conclusions:

• Geo-NI offers a superior approach to light field rendering by synergistically combining NI and DIBR.
• The framework effectively addresses limitations in handling non-Lambertian effects and large disparities.
• Geo-NI represents a significant advancement in geometry-aware light field rendering technology.