DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) excel in computer vision but face computational challenges for efficient deployment.
• Channel pruning reduces model redundancy but existing methods rely on manual rules, potentially limiting performance.
• Large neural networks offer vast pruning potential often untapped by current techniques.

Purpose of the Study:

• To introduce Differentiable Annealing Indicator Search (DAIS), an automated approach for channel pruning in CNNs.
• To leverage Neural Architecture Search (NAS) principles for optimizing pruned models under computational constraints.
• To develop a method that automatically identifies effective channel pruning strategies.

Main Methods:

• DAIS employs bi-level optimization, relaxing binarized channel indicators to continuous values for joint learning with model parameters.
• An annealing-based procedure guides indicator convergence towards binarized states, bridging the continuous-to-discrete gap.
• Regularizations based on structural knowledge are incorporated to control sparsity and enhance performance.

Main Results:

• DAIS automatically searches for optimal pruned models, overcoming limitations of hand-crafted pruning rules.
• The method effectively manages the trade-off between model compression and performance.
• DAIS demonstrates superior performance compared to state-of-the-art pruning techniques on benchmark datasets (CIFAR-10, CIFAR-100, ImageNet).

Conclusions:

• DAIS offers an automated and effective solution for channel pruning in CNNs, enhancing efficiency and performance.
• The proposed method advances the field of neural architecture search for model compression.
• DAIS provides a scalable approach for deploying optimized CNNs in resource-constrained environments.