Scene-Aware Foveated Neural Radiance Fields.

Area of Science:

• Computer Vision
• Computer Graphics
• Virtual Reality

Background:

• Neural Radiance Fields (NeRF) methods offer potential for image synthesis.
• Foveated rendering aims to improve performance by focusing on salient regions.
• Existing NeRF methods struggle with high-quality foveated image synthesis in complex VR scenes.

Purpose of the Study:

• To propose a scene-aware foveated neural radiance fields method for high-quality foveated image synthesis in complex VR scenes.
• To enhance the representation capability of NeRF in salient regions of complex VR scenes.
• To achieve high frame rates for foveated image synthesis.

Main Methods:

• Constructing a multi-ellipsoidal neural representation based on scene content.
• Implementing a uniform sampling based foveated neural radiance field framework.
• Utilizing a foveated scene-aware objective function for improved synthesis quality.

Main Results:

• Synthesized high-quality binocular foveated images at an average of 66 FPS in complex scenes.
• Achieved significantly higher synthesis quality in both foveal and peripheral regions.
• Demonstrated a 1.41-1.46x speedup compared to state-of-the-art foveated NeRF methods.
• User study confirmed high visual similarity to ground truth.

Conclusions:

• The proposed scene-aware foveated NeRF method effectively synthesizes high-quality foveated images in complex VR environments.
• The method achieves substantial improvements in both synthesis quality and rendering speed.
• The approach shows strong potential for real-time VR applications requiring high visual fidelity.