Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) offer superior performance but incur high computational costs.
• Overparameterized neural networks require efficient acceleration methods.
• Existing pruning methods often reduce optimization space and cause information loss.

Purpose of the Study:

• To propose an asymptotic soft filter pruning (ASFP) method for accelerating deep neural network inference.
• To address the limitations of traditional pruning techniques, such as reduced optimization space and information loss.

Main Methods:

• ASFP updates pruned filters during retraining, preserving the original model's optimization space.
• The method employs asymptotic pruning, removing filters gradually during training for stability.
• This approach allows information to concentrate in remaining filters, enhancing training and pruning stability.

Main Results:

• ASFP effectively accelerates deep neural networks on image classification benchmarks.
• On ILSVRC-2012, ASFP reduced ResNet-50 FLOPs by over 40% with only 0.14% top-5 accuracy degradation.
• The proposed method outperformed standard soft filter pruning by 8% in terms of accuracy preservation.

Conclusions:

• ASFP offers an effective strategy for accelerating deep neural networks.
• The method balances computational efficiency with minimal impact on model accuracy.
• ASFP provides a stable and efficient approach to network pruning for practical applications.