Pruning Networks With Cross-Layer Ranking & k-Reciprocal Nearest Filters

Area of Science:

• Computer Vision
• Deep Learning
• Network Pruning

Background:

• Magnitude-based weight pruning methods suffer from a "long-tail" pruning problem.
• Existing filter pruning techniques often involve complex learning processes, increasing computational burden.

Purpose of the Study:

• To propose a novel, efficient, and non-learning based filter-level network pruning method.
• To address the "long-tail" pruning issue and reduce computational complexity in deep neural networks.
• To demonstrate the effectiveness of the proposed method on image classification tasks.

Main Methods:

• Introduced CLR-RNF (Cross-Layer Ranking - Reciprocal Nearest Filter) for filter-level network pruning.
• Developed a computation-aware measurement for individual weight importance.
• Implemented a k-reciprocal nearest filter selection scheme for filter preservation.

Main Results:

• CLR-RNF significantly reduces FLOPs and parameters: 74.1% FLOPs and 95.0% parameters on VGGNet-16 (CIFAR-10).
• Achieved accuracy improvements on CIFAR-10 (0.3%) and minimal drops on ImageNet (1.7% top-five accuracy drop on ResNet-50).
• Demonstrated superiority over state-of-the-art methods in network pruning efficiency and performance.

Conclusions:

• CLR-RNF offers an efficient, non-learning approach to filter-level network pruning.
• The method effectively reduces model size and computational load while maintaining or improving accuracy.
• CLR-RNF presents a promising alternative for optimizing deep neural networks for various applications.