Human activity recognition at a kilometer range using single-photon LiDAR

Human activity recognition has been a prominent research focus for several decades. While computer vision-based passive sensing is desirable for many applications, traditional RGB-based methods face significant limitations, including high computational cost, sensitivity to ambient lighting conditions, and privacy concerns. Single-photon LiDAR (light detection and ranging) is emerging as a robust alternative, offering efficient and high-resolution 3D imaging over long ranges and in difficult conditions while preserving privacy. In this study, we combine an eye-safe single-photon LiDAR system with a deep learning pipeline to achieve fast, long-range human activity recognition. To address the issue of limited available data for training, we generate synthetic datasets by combining real motion capture data with virtual models in a 3D modeling environment. A state-of-the-art recurrent neural network is trained on short-duration depth-image sequences of six activities. We also contribute two long-range, video frame rate single-photon LiDAR datasets for human activity recognition, recorded at distances of 325 m and 1.4 km, which exhibit markedly different noise levels. When evaluated on these data, the network maintains more than 80% accuracy even in the most challenging scenario, while supporting continuous and fast inference.