Multi-way radial consistency pre-training for event based optical flow

Area of Science:

• Computer Vision
• Robotics
• Autonomous Systems

Background:

• Event-based optical flow estimation is crucial for various applications like autonomous driving.
• Existing methods struggle with data scarcity and often ignore rotational/scaling motions.
• Current unsupervised frameworks primarily use 1D temporal reversal for consistency.

Purpose of the Study:

• To develop a self-supervised framework for event-based optical flow estimation that addresses limitations of existing methods.
• To introduce a novel 'radial consistency' loss function for improved motion estimation.
• To enhance optical flow estimation accuracy and computational efficiency, especially in complex scenes with rotational motion.

Main Methods:

• Developed a self-supervised pre-training framework using log-polar coordinates and radial tessellation.
• Introduced the 'radial consistency' loss, a label-free constraint generalizing forward-backward checks to 360°.
• Employed a shared encoder-decoder to predict complementary flow fields with a cyclic sum constraint to zero.
• Optionally applied supervised fine-tuning on small labeled datasets for domain adaptation.

Main Results:

• Achieved state-of-the-art performance on the MVSEC dataset with 0.67 EPE, outperforming E-RAFT (0.89 EPE) and EV-FlowNet (1.10 EPE).
• Demonstrated superior performance on rotation-heavy sequences, achieving 0.93 EPE (fine-tuned) vs. E-RAFT's 1.66 EPE.
• Improved computational efficiency, reaching 55 FPS and 26 GFLOPs, compared to E-RAFT's 42 FPS and 38 GFLOPs.

Conclusions:

• The proposed radial consistency framework offers a robust and efficient solution for event-based optical flow estimation.
• Self-supervised pre-training with radial consistency significantly reduces reliance on large labeled datasets.
• The method achieves competitive accuracy with fully supervised approaches, even with minimal fine-tuning, and shows promise for real-world robotics and automotive applications.