Self-Supervised joint flow and depth estimation via Multi-Cue uncertainty modeling

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Estimating motion and 3D structure from dynamic scenes is crucial for computer vision.
• Self-supervised learning offers a cost-effective alternative to manual annotation but struggles with occlusions and non-rigid motion.
• Existing methods often handle these challenges with separate heuristics, limiting their effectiveness.

Purpose of the Study:

• To develop a unified framework for robust motion and depth estimation in dynamic scenes.
• To leverage task inconsistency as a supervisory signal for self-supervised learning.
• To improve handling of occlusions, texture ambiguity, and non-rigid motion.

Main Methods:

• Proposed UGFD (Uncertainty Guided Flow and Depth) framework.
• Derived dense uncertainty maps by modeling intra-task (gradient disagreements) and inter-task (flow-depth rigidity violations) inconsistencies.
• Introduced Context-Aware Uncertainty (CAU) module and Unrigidity-Driven (URD) loss for guided learning and focused optimization.

Main Results:

• Achieved state-of-the-art performance on KITTI benchmarks.
• Demonstrated robust generalization capabilities through zero-shot tests on Sintel and FlyingThings3D datasets.
• Successfully unified handling of diverse error sources under a consistent uncertainty framework.

Conclusions:

• The proposed uncertainty estimation paradigm effectively addresses limitations in self-supervised motion and depth estimation.
• UGFD framework enables robust estimation without ground truth data by learning to assess confidence.
• This approach offers a significant advancement for computer vision tasks requiring accurate 3D scene understanding.