Joint Sparse Optical Flow Estimation and Keypoint Detection via Dual-task Imperative Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Deep learning for optical flow faces challenges in interpretability, generalization, and efficiency.
• Sparse point tracking is crucial for visual odometry (VO), often more so than dense optical flow.
• Existing methods lack integrated frameworks for joint keypoint detection and sparse optical flow estimation.

Purpose of the Study:

• To develop a novel dual-task imperative learning framework for synergistic sparse optical flow estimation (iFLOW) and adaptive keypoint detection (iPOINT).
• To address limitations in interpretability, generalization, and efficiency in current optical flow models.
• To improve performance in applications like visual odometry through joint optimization.

Main Methods:

• Implemented an Expectation-Maximization (EM) paradigm for alternating optimization of iFLOW and iPOINT.
• Utilized a Gauss-Newton reasoning engine within the EM framework.
• Leveraged convolutional features under the generalized feature invariance principle for an imperative learning mechanism.

Main Results:

• The proposed framework demonstrates enhanced interpretability, cross-domain adaptability, and computational efficiency.
• Ultra-compact models (0.05M parameters for iFLOW, 0.09M for iPOINT) achieved remarkable performance across multiple metrics.
• Significant performance was observed using only 200 training image pairs, outperforming classical and learning-based baselines.

Conclusions:

• The dual-task imperative learning framework effectively integrates keypoint detection and sparse optical flow estimation.
• The proposed approach offers a computationally efficient and highly adaptable solution for visual odometry and related tasks.
• This work advances sparse optical flow estimation by providing interpretable and generalizable models with minimal training requirements.