Light Field Neural Rendering.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Classical light field rendering requires dense sampling for view-dependent effects.
• Geometric reconstruction methods use sparse views but struggle with non-Lambertian effects.

Purpose of the Study:

• To develop a novel neural rendering model combining strengths of light field rendering and geometric reconstruction.
• To accurately represent view-dependent effects and implicitly learn scene geometry from sparse views.

Main Methods:

• Operates on a four-dimensional light field representation.
• Employs a two-stage transformer model aggregating features along epipolar lines and reference views.
• Incorporates geometric constraints during training and inference.
• Introduces modifications for generalization without fine-tuning.

Main Results:

• Achieves state-of-the-art performance on forward-facing and 360° datasets.
• Demonstrates superior performance on scenes with significant view-dependent variations.
• Successfully models view-dependent effects and learns scene geometry from sparse views.

Conclusions:

• The proposed model effectively bridges the gap between dense sampling and sparse view methods.
• It offers a robust solution for novel view synthesis, particularly in complex visual scenarios.
• The approach generalizes well and sets a new benchmark in neural rendering.