Filter Pruning by Switching to Neighboring CNNs With Good Attributes

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Filter pruning is a key technique for reducing computational costs in neural networks.
• Current methods often use static pruning criteria and rely solely on filter magnitude, neglecting inter-filter dependencies.

Purpose of the Study:

• To develop a novel filter pruning method that addresses limitations of existing magnitude-based criteria.
• To improve the efficiency and performance of neural networks through adaptive pruning.

Main Methods:

• Proposed Meta-Attribute-based Filter Pruning (MFP) incorporating geometric filter distances.
• Introduced adaptive criteria selection using a meta-attribute reflecting the network's current state.
• Evaluated the method on image classification benchmarks.

Main Results:

• MFP expands upon magnitude-based pruning by including geometric relationships between filters.
• Adaptive criteria selection optimizes pruning based on the network's dynamic state.
• Achieved over 50% FLOPs reduction on ResNet-50 for ILSVRC-2012 with only 0.44% top-5 accuracy loss.

Conclusions:

• MFP offers a more comprehensive approach to filter importance assessment by considering filter interactions.
• The adaptive meta-attribute strategy enhances pruning effectiveness.
• This method significantly reduces computational overhead while maintaining high accuracy in deep neural networks.