Boosting Photon-Efficient Image Reconstruction With A Unified Deep Neural Network.

Area of Science:

• Computational imaging
• Computer vision
• Deep learning for image reconstruction

Background:

• Photon-efficient imaging utilizes single-photon sensors for 3D image capture, but faces challenges from low photon counts, low signal-to-background ratio (SBR), and multiple returns.
• These limitations hinder reconstruction performance in applications requiring high fidelity under low optical flux.

Purpose of the Study:

• To develop a unified deep neural network capable of simultaneously recovering depth maps and intensity images from photon-efficient measurements.
• To explicitly address the challenges of low SBR and multiple returns in photon-efficient imaging.

Main Methods:

• A novel deep neural network architecture featuring an encoder with a non-local block for spatial-temporal correlation exploitation and two decoders for depth and intensity recovery.
• Integration of a noise prior block informed by background noise photon statistics to enhance reconstruction.
• Training on simulated data with validation on real-world imaging systems.

Main Results:

• The proposed network achieves superior reconstruction fidelity for depth and intensity images, even with extremely low photon counts, low SBR, and significant blur from multiple returns.
• Performance significantly surpasses existing methods in challenging imaging scenarios.
• The network demonstrates strong generalization capabilities from simulated to real-world data.

Conclusions:

• The unified deep neural network effectively overcomes key limitations in photon-efficient imaging, enabling high-quality 3D reconstruction.
• This approach broadens the applicability of photon-efficient imaging in scenarios with strict optical flux constraints.
• The method offers a significant advancement for applications requiring precise imaging under low-light conditions.