Continuous-Time Object Segmentation Using High Temporal Resolution Event Camera.

Area of Science:

• Computer Vision
• Robotics
• Sensor Technology

Background:

• Event cameras offer high temporal resolution and dynamic range, enabling new possibilities for computer vision tasks.
• Segmenting objects from sparse, asynchronous event streams presents unique challenges compared to traditional video data.

Purpose of the Study:

• To develop the first framework for continuous-time object segmentation from event streams.
• To enable accurate object segmentation in challenging conditions with sparse event data.

Main Methods:

• A novel framework incorporating a Recurrent Temporal Embedding Extraction (RTEE) module (ResLSTM) and a Cross-time Spatiotemporal Feature Modeling (CSFM) module (transformer architecture).
• Recurrently processing historical events and masks with current events to continuously update temporal embeddings.
• Construction of real-world and simulated event-based object segmentation datasets.

Main Results:

• Demonstrated effectiveness of the recurrent architecture for continuous-time object segmentation.
• Successful segmentation of complete objects from sparse event streams.

Conclusions:

• The proposed framework effectively addresses the challenges of object segmentation in event streams.
• This work advances the capabilities of event cameras in real-world applications requiring dynamic object tracking and segmentation.