G2-MonoDepth: A General Framework of Generalized Depth Inference From Monocular RGB+X Data

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Monocular depth inference is crucial for robot scene perception but current methods require task-specific fine-tuning for different robots and scenes.
• Existing approaches create burdens for industrialization due to high-cost customization for specific robotic platforms and varied environmental conditions.

Purpose of the Study:

• To develop a unified task and benchmark for monocular depth inference applicable to diverse robotic systems and unseen environments.
• To create a single model capable of inferring high-quality depth maps from varied raw sensor data, accommodating different scales, sparsity, and errors.

Main Methods:

• A unified data representation (RGB+X) was developed to handle diverse inputs including raw depth data with varying sparsity and errors.
• A novel unified loss function was designed to adapt to diverse input data characteristics and output scene scales.
• An improved network architecture and a data augmentation pipeline were implemented to simulate real-world artifacts and ensure robust depth map generation.

Main Results:

• The G2-MonoDepth benchmark demonstrated superior performance across depth estimation, depth completion (varying sparsity), and depth enhancement tasks.
• The unified approach consistently outperformed state-of-the-art baselines on both real-world and synthetic datasets.
• The system effectively handles diverse scene scales, depth sparsity, and sensor errors without task-specific fine-tuning.

Conclusions:

• G2-MonoDepth offers a generalized solution for monocular depth inference, significantly reducing customization costs for large-scale robot deployment.
• The unified framework provides a robust and adaptable method for enhancing robot scene perception capabilities across a wide range of applications.