Exploiting Sparse Self-Representation and Particle Swarm Optimization for CNN Compression.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Structured pruning is crucial for compressing convolutional neural networks (CNNs).
• Existing methods often rely on static parameter statistics and uniform pruning rates, limiting efficiency.
• There's a need for adaptive and scalable pruning techniques.

Purpose of the Study:

• To develop an effective network redundancy elimination approach for CNN compression.
• To enable layer-wise learning of pruning rates for optimal model compression.
• To create a scalable method applicable to various architectures and deeper networks.

Main Methods:

• Constructing a sparse self-representation of filters/neurons to analyze interdependencies.
• Employing particle swarm optimization (PSO) for layer-wise learning of pruning rates.
• Joint optimization during pruning to maintain model performance.

Main Results:

• Significant reduction in computational cost (FLOPs) across different architectures and datasets.
• ResNet50 on ImageNet: 58.1% FLOPs reduction with a 1.6% top-five error increase.
• FCN-ResNet50 on COCO2017: 44.1% FLOPs reduction with a 3% error increase.

Conclusions:

• The proposed method effectively eliminates network redundancy while preserving model accuracy.
• The approach is scalable and adaptable to diverse CNN architectures.
• It outperforms existing state-of-the-art pruning methods in terms of compression and performance.