An Adaptive Refinement Scheme for Depth Estimation Networks.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models excel at depth generation but struggle with generalization and consistent performance on diverse inputs.
• Feature backpropagating refinement (f-BRS) was developed for segmentation to refine predictions using sparse labels, showing potential for depth prediction.
• Directly applying f-BRS to depth prediction resulted in local optima and failed to enhance baseline network performance.

Purpose of the Study:

• To extend the feature backpropagating refinement (f-BRS) scheme for improved depth prediction accuracy in deep learning.
• To develop a novel refinement strategy that overcomes the limitations of existing methods when applied to depth estimation.

Main Methods:

• Proposed a double-stage adaptive refinement scheme (DARS) to enhance depth prediction.
• Stage 1: Incorporated a Delaunay-based correction module to refine depth maps from a baseline network.
• Stage 2: Utilized a particle swarm optimizer (PSO) to fine-tune f-BRS parameters (scales and biases) for precise depth estimation.

Main Results:

• The DARS scheme significantly improved depth estimation compared to baseline networks and the original f-BRS approach.
• The Delaunay-based correction module effectively enhanced initial depth predictions.
• PSO-based fine-tuning of f-BRS parameters further refined the depth maps, achieving superior results.

Conclusions:

• The proposed double-stage adaptive refinement scheme (DARS) is an effective method for improving deep learning-based depth prediction.
• DARS successfully addresses the local optima problem encountered when applying f-BRS to depth estimation.
• The scheme demonstrated effectiveness on both outdoor (KITTI) and indoor (NYUv2) datasets, indicating robust generalization.