Efficient SPAD reconstruction framework with time-frequency domain compression and lightweight joint depth-intensity optimization

The capacity to capture the arrival times of individual photons enables 3D imaging with extreme sensitivity in depth sensing. Existing reconstruction methods directly process the overwhelming output data generated by SPAD arrays, which imposes a heavy burden on computation and storage, exceeding the resource limit of edge devices. To address this problem, this paper proposes an efficient reconstruction framework based on time-frequency domain compression and a lightweight reconstruction neural network with joint depth-intensity optimization. For time-frequency domain compression, the raw histogram is converted to a compact representation by the discrete wavelet transform, frequency-domain peak detection, and multi-scale voting, while retaining sufficient information. For lightweight reconstruction, the dual-branch neural network with joint depth-intensity optimization is proposed, where the depth branch aims to recover depth while the intensity branch targets the depth gradient rather than the intensity or reflectivity image. This specific design is motivated by the fact that ground-truth of the intensity or reflectivity images contain abundant textures that act as non-informative noise for depth reconstruction. By using the depth gradient as the supervision target for the intensity branch, the network successfully captures essential edge details and scene structures while avoiding texture interference. Consequently, this joint optimization enhances the details of the scenes and improves the performance under various illuminances. Additionally, the gradient awareness block and the concatenation fusion block are embedded to extract data associations and facilitate efficient information exchange between the two branches. Experimental results demonstrate that the proposed method achieves competitive reconstruction accuracy (the lowest average RMSE of 0.00921 meters on the Middlebury dataset). Furthermore, the proposed method reduces computations by 72.18% and achieves the minimum latency compared to previous methods, which shows potential for edge-device deployment.