Random pruning: channel sparsity by expectation scaling factor.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural network (DNN) model compression and acceleration are crucial for efficient deployment.
• Existing pruning strategies often involve complex computations and intricate sub-network identification processes.
• Convolutional Neural Networks (CNNs) present opportunities for optimization through structured pruning.

Purpose of the Study:

• To propose a new structured pruning method, EXP, for efficient DNN model compression.
• To address the limitations of existing pruning techniques regarding computational overhead and complexity.
• To leverage the linear relationship between channel matrix elements and expectation scaling ratios for effective pruning.

Main Methods:

• A novel structured pruning method, EXP, is introduced.
• Channels with similar expectation scaling factors ($$) are identified and randomly removed within convolutional layers.
• This approach induces random sparsity, creating non-redundant and non-unique sub-networks.

Main Results:

• EXP achieves significant reduction in Floating Point Operations (FLOPs) across various networks.
• On CIFAR-10, ResNet-56 FLOPs were reduced by 71.9% with only a 0.23% Top-1 accuracy loss.
• On ILSVRC-2012, ResNet-50 FLOPs decreased by 60.0% with a 1.13% Top-1 accuracy loss.

Conclusions:

• The proposed EXP method offers an efficient and effective approach to structured pruning in CNNs.
• EXP successfully reduces model complexity and computational requirements without substantial accuracy degradation.
• The method provides a practical solution for accelerating deep learning models through optimized sub-network generation.