Hierarchical Threshold Pruning Based on Uniform Response Criterion.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) are powerful but often overparameterized, requiring significant memory and training time.
• Filter pruning is an efficient technique to reduce CNN complexity, but existing methods face challenges like over-pruning layers or neglecting inter-layer filter importance.

Purpose of the Study:

• To propose a novel filter pruning method that addresses the limitations of global threshold pruning.
• To introduce a feature-discrimination-based filter importance criterion, the uniform response criterion (URC).
• To develop hierarchical threshold pruning (HTP) integrated with URC for more effective CNN compression.

Main Methods:

• Developed the uniform response criterion (URC) to measure filter importance by analyzing activation response distributions across classes.
• Introduced hierarchical threshold pruning (HTP) to perform pruning within relatively redundant layers, avoiding complete layer pruning.
• Combined URC with score normalization and layer-specific pruning strategies.

Main Results:

• The proposed HTP with URC method effectively prunes filters without sacrificing significant performance.
• Experiments on CIFAR-10/100 and ImageNet datasets demonstrate state-of-the-art results in CNN compression.
• The method successfully avoids over-pruning and accounts for varying filter importance across different layers.

Conclusions:

• Hierarchical threshold pruning (HTP) with the uniform response criterion (URC) offers an effective solution for compressing overparameterized CNNs.
• The proposed approach is suitable for resource-constrained devices by reducing memory and computation.
• This method achieves superior performance compared to existing pruning techniques on benchmark datasets.