Neuromorphic-P²M: processing-in-pixel-in-memory paradigm for neuromorphic image sensors

Area of Science:

• Computer Engineering
• Artificial Intelligence
• Neuromorphic Engineering

Background:

• Edge devices require energy-efficient solutions for processing large sensory data with limited resources.
• In-pixel processing offers high energy efficiency by computing features directly within CMOS image sensors.
• Processing-in-pixel for neuromorphic vision sensors remains an underexplored area.

Purpose of the Study:

• To propose and validate an asynchronous, non-von-Neumann analog processing-in-pixel paradigm for neuromorphic vision sensors.
• To integrate in-situ multi-bit, multi-channel convolution operations within the pixel array.
• To address circuit non-idealities and variations through a hardware-algorithm co-design framework.

Main Methods:

• Developed an analog processing-in-pixel paradigm for convolution using multiply-accumulate (MAC) operations.
• Employed a hardware-algorithm co-design framework incorporating circuit non-idealities, leakage, and process variations.
• Utilized HSpice simulations on GF22nm FD-SOI technology and validated on neuromorphic vision sensor datasets.

Main Results:

• Achieved approximately 2x lower backend-processor energy consumption compared to state-of-the-art methods.
• Maintained comparable front-end (sensor) energy consumption.
• Demonstrated a high test accuracy of 88.36% on the IBM DVS128-Gesture dataset.

Conclusions:

• The proposed analog processing-in-pixel paradigm is a viable and energy-efficient solution for neuromorphic vision sensors.
• This approach significantly reduces power consumption for edge AI, particularly in computer vision tasks.
• The hardware-algorithm co-design framework effectively handles circuit non-idealities for robust performance.