MS-NeRF: Multi-Space Neural Radiance Fields.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Neural Radiance Fields (NeRF) are state-of-the-art for novel view synthesis.
• Standard NeRF methods produce artifacts like blurriness and distortion when rendering scenes with reflective surfaces.
• Handling complex light transport, including reflections and refractions, remains a significant challenge for NeRF.

Purpose of the Study:

• To develop an enhanced NeRF approach capable of accurately rendering scenes with reflective and refractive objects.
• To improve the robustness and quality of NeRF rendering in the presence of challenging optical phenomena.
• To introduce a flexible framework that can be integrated with existing NeRF architectures.

Main Methods:

• Proposed a multi-space neural radiance field (MS-NeRF) framework representing scenes across parallel sub-spaces.
• Developed specialized multi-space modules adaptable to both MLP-based and grid-based NeRF methods.
• Created a novel dataset featuring synthetic and real-world scenes with complex reflections and refractions for comprehensive benchmarking.

Main Results:

• MS-NeRF significantly improved rendering quality on reflective regions, enhancing Mip-NeRF 360 by 4.15 dB in PSNR and TensoRF by 2.71 dB.
• Achieved these improvements with negligible increases in computational overhead (0.5% and 0.046% extra parameters, respectively).
• Demonstrated superior performance over single-space NeRF methods in extensive experiments on complex scenes.

Conclusions:

• The proposed multi-space approach effectively addresses the limitations of existing NeRF methods in handling reflective and refractive objects.
• MS-NeRF offers a computationally efficient enhancement for high-quality scene rendering in challenging visual conditions.
• The novel dataset provides a valuable resource for future research in robust NeRF rendering.