Filter Pruning via Measuring Feature Map Information

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models are computationally intensive, limiting their deployment on resource-constrained devices.
• Existing neural network pruning methods often overlook the relationship between feature maps and filters, focusing solely on filter-intrinsic information.

Purpose of the Study:

• To develop novel pruning methods that efficiently reduce the computational complexity of deep neural networks.
• To explore the utility of feature map information for evaluating filter importance.
• To enhance model compression while maintaining high performance.

Main Methods:

• Proposed a pruning method using information entropy of feature maps to assess filter importance, incorporating normalization for cross-layer comparison.
• Introduced a parallel pruning strategy combining the novel method with slimming techniques for improved computational cost.
• Evaluated methods on ResNet50 (ImageNet) and DenseNet40 (CIFAR10).

Main Results:

• The proposed methods achieved significant reductions in parameters and FLOPs (Floating Point Operations) while preserving accuracy.
• ResNet50 on ImageNet reached 72.02% top-1 accuracy with 11.41M parameters and 1.12B FLOPs.
• DenseNet40 on CIFAR10 achieved 94.04% accuracy with 0.38M parameters and 110.72M FLOPs, further reduced to 0.37M parameters and 100.12M FLOPs with the parallel method.

Conclusions:

• The novel pruning methods effectively reduce computational complexity and resource requirements of deep models.
• Leveraging feature map information provides a valuable approach for filter importance assessment in neural network pruning.
• The proposed techniques offer a superior balance of accuracy, parameter count, and computational cost compared to existing advanced methods.