Continual Adaptation for Deep Stereo.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep neural networks achieve high accuracy in stereo depth estimation by regressing dense disparities.
• Training these networks requires large labeled datasets covering all deployment scenarios, which is often impractical.
• Real-world applications necessitate systems that can adapt to unseen environments and changing conditions.

Purpose of the Study:

• To develop a continual adaptation paradigm for deep stereo networks to address the challenge of unseen environments.
• To introduce a lightweight and modular architecture, Modularly ADaptive Network (MADNet), for efficient adaptation.
• To enable real-time, self-adaptive depth estimation without the need for new labeled data.

Main Methods:

• Proposed Modularly ADaptive Network (MADNet) with Modular ADaptation (MAD, MAD++) algorithms for optimizing independent network parts.
• Utilized self-supervision via right-to-left image warping as a learning signal for online adaptation.
• Incorporated traditional stereo algorithms as an alternative source for learning signals.
• Ensured adaptation relies solely on input images available at deployment time.

Main Results:

• Demonstrated the first real-time self-adaptive deep stereo system.
• Achieved efficient adaptation of independent sub-portions of the network.
• Showcased adaptation using self-supervision or traditional stereo algorithms, requiring no extra data.

Conclusions:

• The proposed MADNet architecture and MAD algorithms facilitate practical deployment of end-to-end deep stereo systems.
• This work pioneers a new paradigm for self-adaptive deep learning models in dynamic environments.
• Enables robust dense disparity regression in challenging and ever-changing real-world scenarios.