Event-based dual photography for transparent scene reconstruction

Area of Science:

• Optics and Photonics
• Computer Vision
• Robotics

Background:

• Light transport is crucial for imaging, with dual photography offering advanced applications.
• Existing dual photography methods face challenges like long acquisition times and large data requirements.
• 3D scanning in non-line-of-sight (NLoS) scenarios remains a significant hurdle in computer vision.

Purpose of the Study:

• To develop a novel hardware setup for efficient dual photography.
• To enable 3D scanning in both line-of-sight (LoS) and NLoS scenes.
• To overcome limitations of current dual photography techniques.

Main Methods:

• A system combining a flying-spot micro-electro mechanical system (MEMS) modulated projector with an event camera was designed.
• The setup facilitates dual photography for capturing light transport information.
• Event-based light transport was utilized for data acquisition and processing.

Main Results:

• The proposed system successfully achieved depth extraction from LoS scenes.
• 3D reconstruction of objects in NLoS scenes was accomplished.
• The novel approach demonstrated improved efficiency and data handling compared to traditional methods.

Conclusions:

• The integrated MEMS projector and event camera system offers a viable solution for advanced dual photography.
• This technology enables robust 3D scanning in challenging LoS and NLoS environments.
• The use of event light transport opens new avenues for real-time 3D reconstruction.