PSE-Net: Channel pruning for Convolutional Neural Networks with parallel-subnets estimator.

Area of Science:

• Deep learning
• Computer vision
• Neural network compression

Background:

• Channel pruning is crucial for compressing deep neural networks.
• Current supernet training strategies are time-consuming due to serial processing.
• Efficiently identifying representative subnets is key for effective pruning.

Purpose of the Study:

• To introduce PSE-Net, a novel parallel-subnets estimator for efficient channel pruning.
• To accelerate supernet training and improve subnet evaluation and ranking.
• To enhance evolutionary search for optimal subnets under resource constraints.

Main Methods:

• Developed a parallel-subnets training algorithm simulating multiple subnet forward-backward passes.
• Utilized feature dropping on the batch dimension for simultaneous subnet training.
• Implemented a prior-distributed-based sampling algorithm to guide evolutionary search.

Main Results:

• Achieved superior supernet training efficiency compared to existing methods.
• Demonstrated improved performance of pruned networks on the ImageNet dataset.
• Pruned MobileNetV2 reached 75.2% Top-1 accuracy under 300M FLOPs, outperforming original by 2.6%.

Conclusions:

• PSE-Net offers significant efficiency gains in supernet training for channel pruning.
• The method effectively identifies high-performing subnets, surpassing state-of-the-art approaches.
• PSE-Net provides a faster and more effective solution for deep neural network compression.