Light-field generation for 3D light-field display with IARF and adaptive ray sampling

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Display Technology

Background:

• Generating high-quality light fields for 3D displays from real scenes is challenging.
• Neural radiance fields have advanced light-field generation but struggle with speed and cylindrical viewpoint synthesis.
• Existing methods face limitations in rendering time and detail for 360-degree scenes.

Purpose of the Study:

• To propose the Instant Adaptive Radiance Field (IARF) method for enhanced light-field generation.
• To improve the quality and reduce the training time for synthesizing high-fidelity 3D light fields.
• To enable realistic rendering of unbounded scene geometry and arbitrary viewpoints.

Main Methods:

• Introduced adaptive ray sampling for resampling points in occupied and unoccupied spaces during ray marching.
• Utilized volumetric sampling consistency (VSC) loss to maintain consistency and reduce training time.
• Employed an iterative network structure for point resampling along rays to ensure convergence.
• Incorporated distortion loss to optimize unbounded scene geometry for realistic rendering.
• Implemented expected viewpoint rendering with backward ray tracing for direct image synthesis.

Main Results:

• The IARF method achieved more detailed viewpoint generation for 360-degree scenes in significantly shorter training times.
• Experimental results showed an average improvement of 2.14 dB in PSNR compared to established methods, with training times around 9 minutes.
• Ablation studies validated the effectiveness of the VSC loss and distortion loss.
• The method successfully generated synthetic images at arbitrary positions and viewing angles, overcoming field-of-view limitations.
• Smooth motion parallax was achieved on a high-resolution 3D light-field display over a large viewing angle.

Conclusions:

• The IARF method effectively enhances synthesized light-field quality and reduces training time for 3D light-field displays.
• The proposed techniques, including adaptive ray sampling and VSC loss, contribute to improved image quality and training efficiency.
• IARF offers a promising solution for real-world scene synthesis in 3D light-field applications.
• The method's ability to render arbitrary viewpoints and achieve smooth motion parallax is crucial for immersive 3D experiences.