Where to Prune: Using LSTM to Guide Data-Dependent Soft Pruning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) excel in vision tasks but face computational and storage limitations on mobile devices.
• Channel pruning, a CNN compression technique, offers high compression rates but requires efficient methods to maintain accuracy.
• Existing pruning methods often focus layer-by-layer, potentially missing global optimization opportunities.

Purpose of the Study:

• To propose a new channel pruning framework for significantly reducing CNN computational complexity.
• To maintain sufficient model accuracy during the compression process.
• To explore a global network pruning scheme rather than layer-by-layer pruning.

Main Methods:

• Utilized a long short-term memory (LSTM) to learn hierarchical network characteristics and generate a global pruning scheme.
• Introduced a data-dependent soft pruning method, Squeeze-Excitation-Pruning (SEP), which selectively excludes kernels during computation.
• Compared soft pruning against traditional hard pruning to better retain model capacity.

Main Results:

• Achieved significant reduction in Floating-point Operations Per Second (FLOPs): 70.1% for VGG and 47.5% for Resnet-56.
• Maintained comparable accuracy to the baseline model despite substantial model compression.
• Demonstrated the effectiveness of the global pruning scheme and soft pruning approach.

Conclusions:

• The proposed channel pruning framework effectively reduces computational complexity and storage overhead in CNNs.
• Soft pruning offers advantages over hard pruning in preserving model performance and knowledge.
• The LSTM-based global pruning scheme enables more efficient network compression for practical deployment.