Laser: Efficient Language-Guided Segmentation in Neural Radiance Fields

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Previous 3D segmentation methods using CLIP features are computationally intensive.
• These methods face challenges with processing speed and storage requirements.

Purpose of the Study:

• To develop an efficient 3D segmentation method using language guidance.
• To streamline the process of 3D scene segmentation through effective CLIP feature distillation.

Main Methods:

• Leveraging CLIP feature distillation for direct and effective processing of dense features.
• Introducing an adapter module and a self-cross-training strategy to mitigate noise.
• Employing a low-rank transient query attention mechanism for accurate segmentation edges.
• Converting segmentation to a classification task via label volume for viewpoint consistency.
• Utilizing a simplified text augmentation strategy to reduce feature-text ambiguity.

Main Results:

• The proposed method achieves efficient and precise 3D segmentation.
• Demonstrates significant improvements in training speed and segmentation performance.
• Enhances segmentation consistency for color-similar areas across different viewpoints.

Conclusions:

• The method offers a streamlined and effective approach to language-guided 3D segmentation.
• It outperforms current state-of-the-art methods in both speed and accuracy.
• The techniques developed address key limitations in previous CLIP-based segmentation approaches.