Baking Neural Radiance Fields for Real-Time View Synthesis.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Neural Radiance Fields (NeRF) excel at creating photorealistic 3D scene representations from images.
• NeRF's high computational cost during rendering limits its application in real-time scenarios.
• Rendering requires numerous queries to a multilayer perceptron (MLP) for each ray.

Purpose of the Study:

• To develop a method for real-time rendering of 3D scenes represented by NeRF.
• To create a compact and efficient scene representation suitable for commodity hardware.
• To overcome the computational limitations of traditional NeRF rendering.

Main Methods:

• Introduced a novel representation called Sparse Neural Radiance Grid (SNeRG).
• Developed a method to precompute and store trained NeRF models into SNeRG.
• Employed a reformulated NeRF architecture and a sparse voxel grid with learned feature vectors.

Main Results:

• SNeRG enables real-time rendering of 3D scenes (over 30 frames per second on a laptop GPU).
• The representation retains NeRF's capability for fine geometric detail and view-dependent appearance.
• SNeRG achieves a compact scene representation, averaging less than 90 MB per scene.

Conclusions:

• Sparse Neural Radiance Grids (SNeRG) offer a viable solution for real-time 3D scene rendering.
• The method significantly reduces computational requirements compared to standard NeRF.
• SNeRG facilitates the deployment of high-fidelity neural volumetric representations on accessible hardware.