Parallelizing analog in-sensor visual processing with arrays of gate-tunable silicon photodetectors

Area of Science:

• Electronics
• Computer Vision
• Materials Science

Background:

• In-sensor processing for computer vision hardware reduces data exchange between sensing and computing units.
• Scalable gate-tunable photodetector arrays are crucial for large-scale in-sensor visual processing.
• Existing systems often require physically separated sensing and computing components.

Purpose of the Study:

• To develop scalable in-sensor visual processing arrays.
• To enable parallelized event sensing and edge detection directly on the sensor.
• To demonstrate a path towards high-throughput computer vision systems.

Main Methods:

• Fabrication of two scalable in-sensor visual processing arrays using dual-gate silicon photodiodes.
• Implementation of CMOS compatible processes for array construction.
• Operation of arrays with zero static power consumption.

Main Results:

• Demonstrated parallelized event sensing and edge detection capabilities.
• Utilized bipolar analog output to capture event-driven light changes and spatial convolution.
• Achieved enhanced performance in classifying dynamic motions and static images.

Conclusions:

• Developed retinomorphic arrays capable of processing both temporal and spatial visual information.
• Showcased the potential of dual-gate silicon photodiodes for in-sensor visual processing.
• Paved the way for large-scale, high-throughput computer vision systems with integrated processing.