Extremely Sparse Networks via Binary Augmented Pruning for Fast Image Classification.

Area of Science:

• Computer Science
• Electrical Engineering
• Artificial Intelligence

Background:

• Network pruning and binarization are key for efficient neural network accelerators.
• Existing methods often compromise accuracy for efficiency, hindering accelerator progress.
• Binary networks offer high efficiency but suffer from a significant accuracy gap compared to full-precision models.

Purpose of the Study:

• To investigate the effectiveness of extremely sparse networks with binary connections for image classification.
• To develop a software-hardware codesign approach for optimizing these networks.
• To achieve a better tradeoff between accuracy and efficiency in neural network accelerators.

Main Methods:

• Proposed a binary augmented extremely pruning method to achieve ~98% sparsity with minimal accuracy degradation.
• Designed a hardware architecture tailored for the resulting sparse and binary networks.
• Conducted experiments on large-scale ImageNet classification and field-programmable gate array (FPGA) platforms.

Main Results:

• The proposed pruning method achieved high sparsity (~98%) with only minor accuracy loss.
• The custom hardware architecture effectively leveraged extreme sparsity and binary connections.
• Demonstrated a prominent tradeoff between accuracy and efficiency on FPGA implementations.

Conclusions:

• Software-hardware codesign is crucial for realizing the potential of extremely sparse and binary neural networks.
• The developed approach significantly improves efficiency while maintaining competitive accuracy for image classification.
• This work advances the design of high-speed and energy-efficient neural network accelerators.