Related Experiment Video
Updated: Oct 12, 2025
20:00
Single Molecule Fluorescence Microscopy on Planar Supported Bilayers
Published on: October 31, 2015
14.1K
High-speed object detection with a single-photon time-of-flight image sensor
Optics Express
|November 23, 2021
Summary
Convolutional neural networks (CNNs) enhance 3D time-of-flight (ToF) imaging for object detection. Using SPAD cameras with CNNs improves depth map resolution and processing speed for real-time applications.
Area of Science:
- Computer Vision
- Photonics
- Machine Learning
Background:
- 3D time-of-flight (ToF) imaging, utilizing single-photon avalanche diodes (SPADs), offers accurate depth data for applications like robotics and augmented reality.
- Current SPAD array limitations in lateral resolution hinder detailed object detection from depth maps.
- High-speed 3D capture is achievable with SPAD arrays and integrated processing, but resolution remains a challenge.
Purpose of the Study:
- To demonstrate the effectiveness of convolutional neural networks (CNNs) in overcoming resolution limitations in SPAD-based 3D ToF imaging for object detection.
- To evaluate the performance of CNNs using full histogram data versus point clouds for object detection.
- To assess the real-time processing capabilities and latency of the proposed system.
Main Methods:
- Development of a portable SPAD camera system capturing 16-bin photon timing histograms with 64x32 spatial resolution.
- Application of convolutional neural networks (CNNs) to process histogram data for object detection.
- Comparison of CNN performance using full histogram data, point clouds, and combined data inputs.
Main Results:
- CNNs significantly improve object detection performance by utilizing full histogram data from SPAD cameras.
- High-speed 3D capture achieved with exposure times as low as 2 ms (500 FPS) and low signal-to-background ratios (0.05).
- GPU-accelerated processing achieves sub-millisecond frame times, resulting in overall system latency in the millisecond range.
Conclusions:
- CNNs effectively address the lateral resolution limitations of SPAD-based 3D ToF imaging for object detection.
- Processing full histogram data or a combination of point cloud and intensity data with CNNs yields high performance.
- The low latency and high-speed capabilities make this approach suitable for safety-critical computer vision applications requiring faster-than-human reaction times.
