Towards an Efficient CNN Inference Architecture Enabling In-Sensor Processing.

Area of Science:

• Computer Vision
• Machine Learning
• Hardware Acceleration

Background:

• Advancements in optical sensing and machine learning enable sophisticated analysis of visual data.
• Convolutional Neural Networks (CNNs) are widely used but computationally intensive, limiting their application on resource-constrained edge devices.
• Processing non-relevant image data creates bottlenecks in real-time edge computing.

Purpose of the Study:

• To develop an efficient, attention-based CNN inference architecture for near-sensor processing.
• To reduce dynamic power consumption and improve real-time processing capabilities on edge devices.
• To overcome computational limitations of traditional CNNs in resource-constrained environments.

Main Methods:

• An attention-based pixel processing architecture designed for near-sensor CNN inference.
• An efficient computation method utilizing hierarchical optimization to reduce convolution operations.
• Exploitation of Spatio-temporal redundancies and relevance scores to minimize computations.
• Hardware implementation on Virtex UltraScale+ FPGA and ASIC (TSMC 90nm).

Main Results:

• Significant reduction in dynamic power consumption for CNN inference.
• High-speed processing capabilities surpassing existing embedded processors.
• Demonstrated feasibility of near-sensor CNN inference on edge devices.
• Effective reduction of computational load by processing only relevant image regions.

Conclusions:

• The proposed attention-based architecture effectively addresses power and speed constraints for edge CNN inference.
• Near-sensor processing with optimized computation significantly enhances efficiency.
• The approach offers a viable solution for deploying advanced machine learning on resource-limited remote sensing platforms.