LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understanding.

Area of Science:

• Computer Vision
• 3D Scene Understanding
• Artificial Intelligence

Background:

• Gaussian Splatting is popular for 3D scene perception.
• Current methods struggle with precise 3D language field alignment, limiting downstream tasks.
• Existing approaches often produce imprecise language fields with outliers.

Purpose of the Study:

• To propose LangSurf, a novel method for accurate 3D language field alignment with object surfaces.
• To enable precise 2D and 3D segmentation using text queries.
• To expand downstream applications like object removal and editing in 3D scenes.

Main Methods:

• LangSurf utilizes a joint training strategy with geometry supervision and contrastive losses.
• Language Gaussians are flattened onto object surfaces for accurate feature assignment.
• A Hierarchical-Context Awareness Module extracts image-level features and uses SAM-segmented masks for fine-grained language features.

Main Results:

• LangSurf achieves precise 2D and 3D segmentation with text queries.
• The method demonstrates superior performance compared to state-of-the-art methods like LangSplat.
• Experiments show effectiveness in instance recognition, removal, and editing tasks.

Conclusions:

• LangSurf significantly improves 3D language field alignment and segmentation accuracy.
• The proposed method enhances capabilities for various 3D scene understanding tasks.
• LangSurf offers a robust solution for precise object manipulation and recognition in 3D environments.