Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation | JoVE Visualize

Area of Science:

Computer Vision
Image Processing
Machine Learning

Background:

Modern optical flow algorithms often rely on sparse descriptors or dense approximate nearest neighbor (ANN) fields for initialization.
Dense ANN fields, while providing dense correspondences, are highly susceptible to outliers as they prioritize visual similarity over accurate flow.
Existing methods often require explicit regularization or smoothing, adding complexity and potential inaccuracies.

Purpose of the Study:

To present a novel dense correspondence field approach for optical flow estimation that is less outlier-prone than traditional ANN fields.
To improve the accuracy and robustness of large displacement optical flow estimation without requiring explicit regularization or new data terms.
To demonstrate the superiority of the proposed method over state-of-the-art descriptor matching techniques in initializing optical flow algorithms.

Main Methods:

Utilizes patch matching techniques and a novel multi-scale matching strategy for dense correspondence field generation.
Incorporates enhancements for outlier filtering within the dense correspondence framework.
Does not rely on explicit regularization, smoothing (e.g., median filtering), or new data terms.

Main Results:

The proposed dense correspondence field approach significantly reduces outliers compared to ANN fields.
When used to initialize the EpicFlow algorithm, the new method outperforms the original state-of-the-art descriptor matching technique.
Significant performance improvements were observed on benchmark datasets including MPI-Sintel, KITTI 2012, KITTI 2015, and Middlebury.

Conclusions:

The novel dense correspondence field method offers a more robust and accurate initialization for large displacement optical flow estimation.
The approach achieves superior performance by effectively handling outliers through advanced matching strategies.
This work provides a foundation for more reliable and efficient optical flow computation in computer vision applications.