E²SL: Efficient Depth Sensing from Event-Based Structured Light.

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Structured light (SL) is crucial for 3D reconstruction but struggles with high-speed dynamic scenes due to frame-based cameras.
• Event cameras offer high temporal resolution, making them promising for high-speed SL imaging.
• Existing event-based SL methods often sacrifice computational efficiency for accuracy, limiting real-world applications.

Purpose of the Study:

• To develop an efficient deep network for monocular event-based structured light (SL) 3D reconstruction.
• To address the trade-off between accuracy and computational efficiency in existing methods.
• To enable robust and fast depth estimation in high-speed dynamic scenarios.

Main Methods:

• Proposed E²SL, an efficient deep network comprising a binary embedding lookup table (BE-LUT), spatial context enhancement (SCE), and geometric-prior regression (GPR).
• BE-LUT efficiently retrieves features from event frames.
• SCE enhances feature receptive fields for spatial context, while GPR uses geometric priors for fast depth estimation.

Main Results:

• Achieved state-of-the-art accuracy in event-based SL.
• Demonstrated a per-frame reconstruction time of 7.7 ms, meeting high-speed depth sensing demands.
• Developed a simulator and collected real-world datasets for training and evaluation.

Conclusions:

• E²SL offers an efficient and accurate solution for monocular event-based SL.
• The method is suitable for real-time, high-speed depth sensing applications.
• The provided datasets and code facilitate further research in event-based 3D reconstruction.