SparseLGS: Sparse View Language Embedded Gaussian Splatting

Area of Science:

• Computer Vision
• 3D Scene Understanding
• Machine Learning

Background:

• Current 3D scene understanding methods often rely on dense multi-view inputs, limiting real-world applications.
• Combining Gaussian Splatting with language embeddings shows promise but requires extensive input data.

Purpose of the Study:

• To develop a method for 3D scene understanding using sparse, pose-free input images.
• To address the limitations of dense input requirements in existing 3D scene representation techniques.

Main Methods:

• Introduced SparseLGS, a method leveraging a learning-based dense stereo model for pose-free, sparse inputs.
• Implemented a three-step region matching approach to resolve multi-view semantic inconsistencies.
• Utilized low-dimensional information extraction and bijections, coupled with a reconstruction loss for improved Gaussian properties.

Main Results:

• SparseLGS achieves comparable semantic field reconstruction quality with significantly fewer inputs (3-4 views) compared to state-of-the-art methods using dense inputs.
• Demonstrated a 5x speedup in computation compared to existing methods using the same sparse input.
• Successfully addressed the 3D semantic field problem with sparse, pose-free inputs, a novel contribution.

Conclusions:

• SparseLGS offers an efficient and effective solution for 3D scene understanding with sparse, pose-free inputs.
• The method reduces computational costs and data requirements, enhancing applicability in real-world scenarios.
• Paves the way for future research in efficient 3D semantic field reconstruction.