Using Algorithmic Transformations and Sensitivity Analysis to Unleash Approximations in CNNs at the Edge.

Area of Science:

• Computer Science
• Artificial Intelligence
• Hardware Acceleration

Background:

• Convolutional Neural Networks (CNNs) exhibit tolerance to arithmetic approximations, but perturbations require careful management to preserve accuracy.
• Current hardware implementation of inexact operators lacks flexibility, often determined at design time without full knowledge of application robustness, leading to inefficient resource allocation.

Purpose of the Study:

• To develop a flexible framework for optimizing CNN models to leverage inexact hardware operations.
• To minimize the hardware cost of Multiply-Accumulate (MAC) operations in CNN inference.

Main Methods:

• A two-phase strategy involving model optimization (bitwidth reduction, error resiliency enhancement) and selective layer assignment to exact or inexact hardware based on a sensitivity metric.
• Incorporation of a highly inexact multiplier design.

Main Results:

• The proposed methodology enables frequent use of inexact operations by enhancing model error resiliency.
• Significant cost reduction of MAC operations (up to 83.6%) in CNN inference was achieved within a 5% accuracy degradation threshold.
• Comparison against state-of-the-art optimized exact implementations demonstrates substantial hardware cost savings.

Conclusions:

• The developed framework effectively balances the use of inexact hardware with maintaining CNN accuracy.
• This approach offers a practical solution for designing energy-efficient and cost-effective hardware for deep learning inference.