Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians.

Area of Science:

• Computer Vision
• Computer Graphics
• Geometric Deep Learning

Background:

• 3D Gaussian Splatting (3D-GS) offers high-fidelity and efficient scene representation but faces challenges in large-scale environments due to excessive Gaussian primitives.
• Rendering all primitives, irrespective of projected size in zoomed-out views, leads to inefficient capacity utilization and poor detail capture at multiple scales.

Purpose of the Study:

• To introduce Octree-GS, a novel Level-of-Detail (LOD) structured approach for efficient rendering of large-scale scenes using Gaussian primitives.
• To improve the scalability and performance of Gaussian-based scene representations.

Main Methods:

• Developed an LOD-structured approach that dynamically selects multi-scale Gaussian primitives for consistent rendering performance.
• Implemented an innovative grow-and-prune strategy for Gaussian densification and a progressive training strategy for LOD arrangement.
• Demonstrated the generalizability of the LOD strategy to other Gaussian-based methods like 2D-GS and Scaffold-GS.

Main Results:

• Octree-GS achieves real-time rendering speeds, up to 10x faster than state-of-the-art methods in large-scale scenes.
• The method maintains high scene reconstruction accuracy and visual quality.
• Reduced the number of primitives required for rendering without performance degradation.

Conclusions:

• Octree-GS effectively addresses the scalability limitations of 3D-GS in large scenes through an efficient LOD strategy.
• The proposed method offers significant speedups and maintains visual fidelity, making it suitable for complex scene rendering.
• The LOD framework is adaptable to various Gaussian-based rendering techniques.