Ultralow Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel...

Area of Science:

• Neuromorphic Engineering
• Integrated Photonics
• Artificial Intelligence Hardware

Background:

• Traditional computing architectures (von Neumann) face limitations in power and speed due to data movement and conversions.
• Current in-sensor computing often relies on tunable sensors or weighting elements for linear operations.
• Efficient, low-power processing at the sensor level is crucial for advanced AI and edge computing applications.

Purpose of the Study:

• To implement in-sensor computing using oscillatory neuron devices for parallel, nonlinear operations.
• To demonstrate image processing functions and neural network inference directly on a focal plane array.
• To project the energy efficiency of this novel in-sensor computing approach.

Main Methods:

• Utilized coupled oscillatory retinal neuron devices converting optical signals to voltage oscillations.
• Developed a computing scheme based on frequency shifts of coupled oscillators for parallel, frequency-multiplexed nonlinear operations.
• Experimentally implemented a 3x3 focal plane array for image processing tasks and MNIST digit recognition.

Main Results:

• Demonstrated parallel execution of edge detection, thresholding, and segmentation functions on the 3x3 array.
• Successfully performed experimental inference on handwritten digits from the MNIST database using the neuron array.
• Projected ultra-low energy consumption for image processing operations, potentially as low as 15 aJ/OP.

Conclusions:

• In-sensor computing with coupled oscillatory neurons enables efficient, parallel, nonlinear computations directly at the sensor.
• This approach offers significant advantages in power and speed for image processing and AI inference.
• The technology shows promise for next-generation low-power, high-performance edge computing systems.