Self-supervised cross frame-rate event-frame depth learning without ground truth.

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Event-based cameras offer high temporal resolution and dynamic range, ideal for challenging environments.
• Existing depth estimation methods struggle with annotation dependency, temporal instability, and inefficient sensor fusion.

Purpose of the Study:

• To propose a novel Self-Supervised Event-Frame Self-Attention Transformer (EF-SAT) for accurate depth estimation.
• To overcome limitations of current self-supervised methods in event-frame data processing and fusion.

Main Methods:

• A Vision Transformer-based encoder jointly models event streams and intensity frames.
• A cross-frame-rate self-attention mechanism captures spatiotemporal dependencies.
• Self-supervised learning utilizes photometric reconstruction, event-frame alignment, temporal consistency, and smoothness constraints.

Main Results:

• The EF-SAT framework achieves superior performance on the DAVIS-240C dataset.
• Attained an Absolute Relative Error of 0.114 and RMSE of 0.163, outperforming state-of-the-art baselines.
• Demonstrates robust, temporally stable, and accurate depth estimation.

Conclusions:

• The proposed EF-SAT model offers a scalable and efficient solution for self-supervised depth estimation.
• Highlights potential for real-world applications in robotics, autonomous navigation, and event-driven perception.