Photon-efficient camera with in-sensor computing

Area of Science:

• Optoelectronics
• Computer Vision
• Quantum Technology

Background:

• Image sensors with integrated computing reduce power and latency in machine vision.
• In-sensor computing at the single-photon level remains an unachieved goal.
• 2D semiconductors offer tunable properties for in-sensor computing applications.

Purpose of the Study:

• To demonstrate a photon-efficient camera with in-sensor computing capabilities.
• To achieve single-photon level in-sensor computing for enhanced machine vision.
• To explore the potential of superconducting nanowire detectors for low-light imaging.

Main Methods:

• Developed a superconducting nanowire array detector with four programmable dimensions: photon count rate, response time, pulse amplitude, and spectral responsivity.
• Achieved saturated (100%) quantum efficiency across a broad spectral range (405-1550 nm).
• Utilized multidimensional modulation and ultra-high sensitivity for data processing.

Main Results:

• Demonstrated a classification accuracy of 92.22% for three letters using only 0.12 photons per pixel per pattern.
• Successfully performed image preprocessing and spectral classification within the sensor.
• Showcased the feasibility of in-sensor computing at the single-photon level.

Conclusions:

• The photon-efficient camera enables high-sensitivity vision tasks in extremely low-light conditions.
• This technology is suitable for applications like covert imaging, biological imaging, and space exploration.
• The single-photon image sensor architecture can be scaled for complex neural networks and real-time vision tasks.