Temporal Stereo Matching From Event Cameras via Joint Learning With Stereoscopic Flow

Area of Science:

• Computer Vision
• Robotics
• Sensor Technology

Background:

• Event cameras, inspired by the retina, offer high dynamic range, temporal resolution, and low power consumption.
• They excel in perceiving 3D environments under challenging conditions due to continuous, detailed pixel movement recording.
• Leveraging the temporal density of event data is crucial for advanced perception tasks.

Purpose of the Study:

• To develop a temporal event stereo framework that utilizes past information for enhanced 3D environment perception.
• To improve stereo matching accuracy by integrating temporal data from event cameras.
• To demonstrate the computational efficiency of the proposed method.

Main Methods:

• Introduced a temporal event stereo framework integrating past event data.
• Jointly trained an event stereo matching network with stereoscopic flow.
• Trained motion flows using disparity maps instead of optical flow ground truth.
• Employed cascading stacking of past data for computational efficiency.

Main Results:

• Achieved state-of-the-art stereo matching performance on MVSEC, DSEC, M3ED, and EVIMO2 datasets.
• Demonstrated significant performance boost through temporal aggregation of information via stereoscopic flow.
• Showcased computational efficiency in processing event data.

Conclusions:

• The temporal event stereo framework effectively leverages past information for robust 3D perception.
• The method offers a significant advancement in event-based stereo matching.
• This approach presents a computationally efficient solution for real-time applications.