NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields.

Area of Science:

• Computer Vision
• Virtual Reality
• Neural Rendering

Background:

• Real-time exploration of dynamic 4D spatiotemporal scenes in VR is challenging, especially with limited camera input.
• Existing methods struggle with efficient reconstruction, modeling, and streamable rendering for dynamic environments.

Purpose of the Study:

• To develop an efficient framework for fast reconstruction, compact modeling, and streamable rendering of dynamic 4D scenes.
• To enable visually exploring real-world dynamic scenes in VR using few or single RGB cameras.

Main Methods:

• Decomposition of 4D spatiotemporal space based on temporal characteristics (static, deforming, new areas).
• Representation and regularization of each area using separate neural fields.
• Hybrid representations based feature streaming for efficient neural field modeling (NeRFPlayer).

Main Results:

• NeRFPlayer achieves fast reconstruction (10 seconds per frame) and interactive rendering.
• Evaluated on dynamic scenes from single and multi-camera arrays, it shows comparable or superior performance to state-of-the-art methods.
• Demonstrates efficient modeling and streamable rendering capabilities.

Conclusions:

• NeRFPlayer offers an efficient solution for real-time 4D dynamic scene exploration in VR.
• The framework effectively handles dynamic scenes captured with limited camera setups.
• Achieves high-quality reconstruction and rendering speeds suitable for interactive applications.