A Lightweight Optical Flow CNN -Revisiting Data Fidelity and Regularization.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Variational methods have traditionally dominated optical flow estimation.
• Recent advancements utilize Convolutional Neural Networks (CNNs), with state-of-the-art models like FlowNet2 requiring substantial parameters.
• Existing CNNs, while promising, face challenges in efficiency and model size.

Purpose of the Study:

• To develop a more efficient and accurate optical flow estimation method using CNNs.
• To reduce model size and increase running speed compared to existing state-of-the-art models.
• To integrate principles from variational methods into a lightweight CNN architecture.

Main Methods:

• Introduced LiteFlowNet2, a lightweight cascaded CNN for optical flow estimation.
• Employed a spatial-pyramid formulation with early correction and descriptor matching.
• Utilized feature-driven local convolutions for flow regularization and feature warping for pyramidal feature extraction.

Main Results:

• LiteFlowNet2 significantly outperforms FlowNet2 in accuracy on Sintel and KITTI benchmarks.
• Achieved a 25.3x reduction in model size and a 3.1x increase in running speed compared to FlowNet2.
• Demonstrated substantial accuracy improvements over LiteFlowNet, with notable gains on Sintel and KITTI datasets.

Conclusions:

• LiteFlowNet2 presents a highly efficient and accurate solution for optical flow estimation.
• The network's design effectively combines variational method principles with deep learning.
• Publicly available code and models facilitate further research and application in computer vision.