DPSNet: Multitask Learning Using Geometry Reasoning for Scene Depth and Semantics.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Monocular depth estimation and semantic understanding are challenging computer vision tasks.
• Existing joint learning frameworks often fail to model geometric structures due to limitations in learning camera motion.
• Accurate scene understanding requires integrating depth, camera pose, and semantic information.

Purpose of the Study:

• To propose DPSNet, a multitask learning method for joint depth estimation, camera pose estimation, and semantic scene segmentation from monocular images.
• To address limitations in existing methods by incorporating geometric reasoning and camera motion learning.
• To improve the accuracy and robustness of scene understanding in computer vision.

Main Methods:

• Developed DPSNet, a novel multitask learning architecture.
• Introduced a rigid semantic consistency loss for robust depth and camera pose prediction, overcoming limitations of pixel reconstruction.
• Utilized multiscale geometric reasoning for accurate semantic scene segmentation.

Main Results:

• DPSNet demonstrated state-of-the-art performance across all three tasks: depth estimation, camera pose estimation, and semantic scene segmentation.
• Experiments on open-source and custom datasets validated the effectiveness of each component of DPSNet.
• The proposed rigid semantic consistency loss proved effective in handling moving pixels and improving geometric modeling.

Conclusions:

• DPSNet offers a significant advancement in multitask learning for monocular vision tasks.
• The integration of geometric reasoning and semantic consistency enhances scene understanding capabilities.
• The model's state-of-the-art performance highlights its potential for real-world applications in autonomous driving and robotics.