Carrying Out CNN Channel Pruning in a White Box

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) are computationally intensive, necessitating compression techniques like channel pruning.
• Existing channel pruning methods often lack interpretability, overlooking individual channel contributions to classification.
• This research addresses the need for explainable channel pruning in CNNs.

Purpose of the Study:

• To develop an interpretable channel pruning method for CNNs.
• To leverage channel contribution analysis for effective model compression.
• To improve the efficiency and accuracy of CNNs through explainable pruning.

Main Methods:

• Channel pruning is performed in a "white box" manner, analyzing feature map visualizations.
• A class-wise mask is dynamically trained to model each channel's contribution to category differentiation.
• A global voting mechanism, based on learned masks, identifies and removes less discriminative channels, followed by fine-tuning.

Main Results:

• The proposed White-Box method demonstrates superior performance compared to state-of-the-art techniques.
• On CIFAR-10, ResNet-110 achieved a 65.23% FLOP reduction with a 0.62% accuracy improvement.
• On ILSVRC-2012, ResNet-50 saw a 45.6% FLOP reduction with only a 0.83% top-1 accuracy loss.

Conclusions:

• Incorporating CNN interpretability theory into channel pruning offers significant advantages.
• White-Box channel pruning effectively compresses CNNs while preserving or enhancing classification accuracy.
• This interpretable approach represents a novel direction in efficient deep learning model design.