Deep learning-enhanced super-resolution imaging using low-cost single photon avalanche diodes.

Area of Science:

• Computer Vision
• Photonics
• Artificial Intelligence

Background:

• Low-cost Single-Photon Avalanche Diode (SPAD) arrays offer potential for depth sensing.
• Existing super-resolution (SR) methods often require complex fusion or significant computational resources.
• Enhancing the performance of consumer-grade SPAD arrays is crucial for broader adoption.

Purpose of the Study:

• To develop a non-fusion deep learning (DL) based super-resolution (SR) solution for low-cost SPAD arrays.
• To reconstruct high-resolution (HR) depth and intensity images from low-resolution (LR) inputs.
• To optimize the DL model for efficient hardware deployment and real-time applications.

Main Methods:

• A compact deep learning (DL) model was designed to process low-resolution (LR, 8x8) depth and intensity data.
• The model simultaneously reconstructs high-resolution (HR, 50x50) images.
• Model compression via INT8 quantization was performed to facilitate hardware deployment.
• Evaluation was conducted on synthetic datasets and real measurements from an STMicroelectronics VL53L8CX SPAD array.

Main Results:

• The DL model demonstrated high fidelity against ground truth images on synthetic datasets.
• Precise structural details were achieved in reconstructions from real SPAD array measurements.
• INT8 quantization resulted in only marginal accuracy loss while enabling hardware deployment.
• Both original and quantized models achieved video-rate SR reconstruction on a mid-range GPU.

Conclusions:

• The proposed compact DL model effectively enhances the performance of low-cost SPAD arrays for super-resolution depth imaging.
• The model's efficiency and accuracy make it suitable for real-time applications.
• Its compact size and potential for hardware deployment position it well for edge-computing, mobile, and embedded systems.