Resource-Efficient and Layer Interdependence-Aware CNN Pruning Leveraging Filter Replacement.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Traditional convolutional neural network (CNN) pruning methods often use heuristic criteria, leading to inconsistent performance and limited generalizability.
• Existing pruning techniques may lack efficiency and adaptability for modern deep learning models.

Purpose of the Study:

• To introduce a novel filter replacement (FR) framework for CNN pruning, treating pruning as replacing filters with zero filters.
• To develop an efficient, data-free pruning algorithm with low complexity by deriving an error bound and defining a submodular importance function.
• To extend the FR framework with optimal nonzero filter replacements and introduce a resource efficiency (RE) metric.

Main Methods:

• Proposed a filter replacement (FR) framework for CNN pruning.
• Derived an upper bound on the absolute error to define an efficient, $\gamma $-weakly submodular importance function.
• Developed a data-free oblivious algorithm for filter selection and extended FR with best-approximation techniques for filter replacements.
• Introduced a resource efficiency (RE) metric to evaluate pruning methods.

Main Results:

• Achieved state-of-the-art results on benchmark networks and datasets.
• Demonstrated a 25.5% reduction in network parameters for ResNet-50 on ImageNet, improving accuracy from 75.15% to 76.52%.
• The layer interdependence-aware pruning (LIAP) method showed up to $10^{11}$ times greater efficiency compared to existing techniques.

Conclusions:

• The proposed filter replacement (FR) framework offers an effective and efficient approach to CNN pruning.
• The data-free, low-complexity algorithm and optimized filter replacements achieve superior performance and resource efficiency.
• This work sets a new standard for resource-aware CNN pruning, balancing accuracy and model compression.