Imaging hidden objects with consumer LiDAR via motion-induced sampling

Light-detection and ranging (LiDAR) is being increasingly deployed for consumer imaging across handheld, wearable and robotic applications^1-4. These sensors measure the time-of-flight of light at picosecond resolution, which could enable them to image objects hidden from their field of view. Although such non-line-of-sight (NLOS) imaging capabilities have been shown on research-grade LiDAR devices, they remain challenging to achieve on consumer devices due to poor signal quality resulting from low laser power, low spatial resolution, and object and camera motion. Here we propose a multi-frame fusion strategy to overcome these challenges and demonstrate NLOS imaging on consumer LiDAR. We introduce the motion-induced aperture sampling model to unify the effects of object shape, object motion and camera motion under a single measurement model. Using this model, we demonstrate several NLOS capabilities on a smartphone-grade LiDAR: (1) three-dimensional reconstruction; (2) single- and multi-object tracking; and (3) camera localization using hidden objects. Previously, NLOS imaging capabilities were restricted to bulky and expensive research-grade hardware that requires extensive set-up and calibration. Our results represent a shift towards plug-and-play NLOS imaging, where anyone can image hidden objects with off-the-shelf hardware (for less than US$100) and no additional set-up. We believe democratization of such capabilities will advance consumer applications of NLOS imaging.