NIS-SLAM: Neural Implicit Semantic RGB-D SLAM for 3D Consistent Scene Understanding

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Neural implicit representations are gaining traction in Simultaneous Localization and Mapping (SLAM).
• Existing SLAM approaches often lack robust scene understanding capabilities.
• Semantic information integration remains a challenge in dense RGB-D SLAM.

Purpose of the Study:

• To develop an efficient neural implicit semantic RGB-D SLAM system (NIS-SLAM) for improved scene understanding.
• To leverage pre-trained 2D segmentation networks for consistent semantic representations in SLAM.
• To enhance surface reconstruction and spatial understanding using novel implicit scene representations.

Main Methods:

• Combines high-frequency multi-resolution tetrahedron-based features and low-frequency positional encoding for implicit scene representation.
• Proposes a semantic probability fusion strategy across non-keyframes and keyframes for consistent semantic learning.
• Implements confidence-based pixel sampling and progressive optimization for robust camera tracking.

Main Results:

• NIS-SLAM demonstrates competitive or superior performance compared to existing neural dense implicit RGB-D SLAM methods.
• Achieves high-fidelity surface reconstruction and spatially consistent scene understanding.
• Validated through extensive experiments on diverse datasets.

Conclusions:

• NIS-SLAM effectively integrates semantic understanding into neural implicit SLAM.
• The proposed fusion strategy enhances semantic consistency.
• The system shows potential for real-world applications like augmented reality.