Self-supervised depth super-resolution with contrastive multiview pre-training.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Low-level vision tasks like guided depth super-resolution (GDSR) face challenges due to limited paired training data.
• Self-supervised learning offers a solution, but upsampling depth maps without high-resolution targets remains difficult.

Purpose of the Study:

• To propose a novel self-supervised depth super-resolution method.
• To address the challenge of insufficient paired training data in GDSR.
• To improve the accuracy and generalization of depth map upsampling.

Main Methods:

• A self-supervised depth super-resolution approach utilizing contrastive multiview pre-training.
• A strategy adaptable for regression tasks, even with small datasets, reducing information redundancy by extracting unique guide features.
• A novel mutual modulation scheme for computing local spatial correlation between cross-modal features.

Main Results:

• The proposed method achieves superior performance compared to state-of-the-art GDSR techniques.
• Demonstrates effective depth map upsampling without explicit high-resolution supervision.
• Exhibits good generalization capabilities across different modalities.

Conclusions:

• The developed self-supervised method effectively overcomes data limitations in GDSR.
• The contrastive multiview pre-training and mutual modulation scheme enhance feature extraction and spatial correlation.
• The approach shows significant potential for advancing self-supervised depth estimation and related vision tasks.