Secrets of Event-Based Optical Flow, Depth and Ego-Motion Estimation by Contrast Maximization

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Event cameras offer high dynamic range and capture scene dynamics efficiently.
• Existing event-based optical flow methods often adapt frame-based approaches, requiring significant modifications.
• There is a need for principled methods tailored to the unique properties of event data.

Purpose of the Study:

• To develop a novel method for dense optical flow, depth, and ego-motion estimation using only event camera data.
• To extend the Contrast Maximization framework for event-based vision tasks.
• To address challenges in event data processing, such as alignment and occlusions.

Main Methods:

• A principled extension of the Contrast Maximization framework for event data.
• Modeling space-time properties of event data and tackling event alignment.
• Designing an objective function to prevent overfitting, handle occlusions, and improve convergence with a multi-scale approach.

Main Results:

• Achieved state-of-the-art unsupervised performance on the MVSEC benchmark and competitive results on the DSEC benchmark.
• Enabled simultaneous dense depth and ego-motion estimation from events.
• Demonstrated effectiveness in unsupervised learning settings and highlighted limitations of current benchmarks.

Conclusions:

• The proposed method provides a robust and principled approach for motion-related tasks using event cameras.
• It offers a strong foundation for future research in event-based vision.
• The method shows potential for various downstream applications in robotics and autonomous systems.