Implicit and Explicit Regularization for Optical Flow Estimation

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Monocular optical flow estimation is crucial for understanding motion from single images.
• Existing methods struggle with flow leakage across objects and maintaining motion consistency within rigid objects.
• Exploiting contextual information is key to overcoming these limitations.

Purpose of the Study:

• To propose two novel and practical regularizing methods for neural networks used in monocular optical flow estimation.
• To address deficiencies like flow leakage and motion inconsistency by leveraging semantic and spatial information.
• To improve the accuracy, stability, and efficiency of optical flow estimation.

Main Methods:

• Introduced a semantic regularization method using semantic segmentation masks and a novel loss function to penalize motion inconsistency at object boundaries.
• Developed a spatial regularization method by incorporating pixel coordinates as additional input features to enhance training stability and efficiency.
• Combined both regularization methods to achieve synergistic performance improvements.

Main Results:

• The semantic regularization method effectively identifies object edges and improves local motion flow reasoning.
• The spatial regularization method enforces consistent flow, improves performance, and reduces convergence time.
• The combined approach demonstrated significant quantitative and qualitative improvements on benchmark datasets, outperforming existing state-of-the-art architectures.

Conclusions:

• The proposed regularization methods are effective, architecture-agnostic, and can be integrated without adding inference complexity.
• Leveraging semantic and spatial information provides complementary benefits for optical flow estimation.
• These novel techniques offer a practical solution for enhancing neural network performance in monocular optical flow tasks.