SeG-Gaussian:Segmentation-Guided 3D Gaussian Optimization for Novel View Synthesis.

Area of Science:

• Computer Vision
• Computer Graphics
• Geometric Modeling

Background:

• Novel view synthesis has been revolutionized by radiance field methods.
• 3D Gaussian Splatting (3DGS) offers real-time rendering with high-fidelity results.
• 3DGS faces challenges in sparsely observed regions due to insufficient gradient information for optimization.

Purpose of the Study:

• To address the limitations of 3DGS in sparsely observed regions.
• To enhance the detail and accuracy of 3D scene reconstruction.
• To improve the spatial distribution and representation of 3D Gaussians.

Main Methods:

• Leveraging segmentation information to identify poorly optimized regions in 3DGS.
• Applying split and clone operations on 3D Gaussians in identified regions.
• Introducing spatial regularization terms: repulsion loss and smoothness loss.

Main Results:

• Achieved state-of-the-art performance in real-time novel view synthesis.
• Significantly improved visibility in sparsely observed regions.
• Resulted in a more compact and accurate 3D scene representation.

Conclusions:

• The proposed method effectively refines 3D Gaussian representations by addressing sparse regions.
• Spatial regularization minimizes Gaussian overlap and redundancy, enhancing geometric accuracy.
• The approach leads to superior novel view synthesis and detailed 3D reconstructions.