Monocular Object-Level SLAM Enhanced by Joint Semantic Segmentation and Depth Estimation.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Simultaneous Localization and Mapping (SLAM) is crucial for mobile robots and augmented reality (AR).
• Monocular SLAM systems face challenges with scale ambiguity and tracking in dynamic environments using only RGB images.
• Integrating semantic information can enhance SLAM by mimicking human visual perception.

Purpose of the Study:

• To develop a monocular object-level SLAM system that overcomes limitations of traditional RGB-based approaches.
• To enhance SLAM performance by incorporating real-time joint depth estimation and semantic segmentation.
• To improve accuracy and robustness in scale determination and dynamic scene tracking.

Main Methods:

• Proposed JSDNet, a multi-task network for simultaneous depth estimation and semantic segmentation.
• Implemented feature fusion and semantic consistency optimization within JSDNet.
• Developed an object-level SLAM system integrating pixel-level and object-level semantics.
• Integrated a scale recovery process for accurate scale evaluation.

Main Results:

• JSDNet achieved state-of-the-art depth estimation and high segmentation precision in real-time on the NYU depth v2 dataset.
• The object-level SLAM system demonstrated reduced trajectory errors on the TUM RGB-D dataset compared to existing SLAM systems.
• The system effectively addressed scale ambiguity and improved tracking in challenging scenarios.

Conclusions:

• The proposed monocular object-level SLAM system significantly enhances localization and mapping capabilities.
• Joint depth estimation and semantic segmentation provide robust semantic understanding for SLAM.
• The system offers a promising solution for real-time applications in mobile robotics and AR.