SNI-SLAM++: Tightly-Coupled Semantic Neural Implicit SLAM

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Simultaneous Localization and Mapping (SLAM) systems traditionally struggle with semantic understanding and robust performance in challenging environments.
• Integrating geometric and semantic information is crucial for advanced scene comprehension and real-world applications.

Purpose of the Study:

• To develop a tightly-coupled semantic SLAM system, SNI-SLAM++, that enhances mapping accuracy, surface reconstruction, and camera tracking.
• To leverage neural implicit representations for a more comprehensive understanding of the environment by integrating visual appearance, geometry, and semantics.

Main Methods:

• Introduction of hierarchical semantic representation for multi-level scene comprehension.
• Integration of appearance, geometry, and semantic features via cross-attention for robust environmental understanding.
• Development of a semantics-coupled tracking framework with semantic loop closure optimization for pose estimation.

Main Results:

• SNI-SLAM++ demonstrates superior mapping and tracking accuracy compared to state-of-the-art visual SLAM methods on benchmark datasets (Replica, ScanNet, TUM-RGBD, ScanNet++).
• The system exhibits excellent capabilities in accurate semantic segmentation and 3D semantic mapping.
• The proposed feature loss effectively guides network optimization at a higher level than traditional RGB or depth losses.

Conclusions:

• SNI-SLAM++ offers a robust and accurate solution for semantic SLAM by tightly integrating multi-modal environmental information.
• The system's novel components, including hierarchical semantic representation and cross-attention, contribute to its enhanced performance and robustness.
• This work advances the field of semantic SLAM, paving the way for more sophisticated scene understanding in robotic applications.