KalmanFlow 2.0: Efficient Video Optical Flow Estimation via Context-Aware Kalman Filtering.

Area of Science:

• Computer Vision
• Machine Learning
• Signal Processing

Background:

• Traditional optical flow methods focus on single image pairs, neglecting temporal dynamics in video sequences.
• Estimating multiple consecutive flow fields in videos presents challenges due to noise and scene complexities.

Purpose of the Study:

• To develop an efficient method for video optical flow estimation that leverages temporal coherence and context dynamics.
• To improve the robustness of optical flow estimation in challenging scenarios like lighting changes and occlusions.

Main Methods:

• Formulating pixel motion flow as a second-order time-variant state vector within a Kalman filtering system.
• Optimally estimating flow fields using maximum a posteriori criteria, considering measurement and system noise.
• Evaluating measurement noise via temporal derivatives, spatial gradients, and warping errors; determining system noise using contextual features from convolutional neural networks.

Main Results:

• The proposed context-aware Kalman filtering method demonstrates improved robustness against abrupt lighting changes and occlusion/dis-occlusion.
• Experimental results on MPI Sintel, Monkaa, and Driving datasets show favorable performance compared to state-of-the-art approaches.
• The method effectively utilizes temporal coherence and context dynamics for accurate multi-frame optical flow estimation.

Conclusions:

• The proposed Kalman filtering approach offers an efficient and robust solution for video optical flow estimation.
• Exploiting temporal coherence and context-aware noise evaluation enhances flow field accuracy in complex video sequences.
• This method advances the state-of-the-art in estimating multiple consecutive flow fields for video analysis.