Model pruning based on filter similarity for edge device deployment.

Area of Science:

• Deep Learning
• Computer Vision
• Model Compression

Background:

• Filter pruning is crucial for accelerating deep learning inference and hardware compatibility.
• Existing criterion-based pruning methods often ignore edge-feature filters and suffer from correlated criteria, leading to suboptimal pruning structures.

Purpose of the Study:

• To propose a novel filter pruning method that addresses limitations of current importance-based criteria.
• To develop an effective and simple pruning strategy based on filter similarity for efficient network compression.

Main Methods:

• Calculates pairwise filter similarity within convolutional layers to obtain a similarity distribution.
• Prunes filters with high similarity to others, either by deletion or setting to zero.
• Employs iterative pruning strategies (hard and soft) for accuracy-memory trade-offs, without layer-specific pruning rates.

Main Results:

• Achieved 61.1% FLOPs reduction and 58.3% parameter reduction with no Top-1 accuracy loss on ResNet-56 (CIFAR10).
• Reduced 53.05% FLOPs on ResNet-50 (ILSVRC-2012) with only 0.29% Top-1 accuracy degradation.
• Demonstrated effectiveness across diverse datasets and network architectures.

Conclusions:

• The proposed filter similarity-based pruning method is effective for deep learning model compression.
• This approach offers significant reductions in FLOPs and parameters while maintaining high accuracy.
• The method provides flexibility for different accuracy-memory trade-offs and simplifies the pruning process.