MLPruner: pruning convolutional neural networks with automatic mask learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) face challenges with computational complexity and large parameter counts.
• Current filter pruning methods rely on heuristics or weight penalties, which can be suboptimal or interfere with training.

Purpose of the Study:

• To develop an autonomous filter pruning method for CNNs that overcomes limitations of existing approaches.
• To introduce a mask learning technique that efficiently identifies filters for pruning without negatively impacting neural network training.

Main Methods:

• A novel mask learning approach is proposed, assigning a learnable mask to each filter in the CNN.
• Masks are converted to binary values during forward propagation to indicate pruning necessity.
• The straight-through estimator (STE) is employed during backward propagation to handle the non-differentiable nature of mask gradients.

Main Results:

• Learned masks accurately reflect the importance of individual filters.
• The proposed method, MLPruner, effectively prunes filters without interfering with the training of neural network weights.
• Demonstrated efficacy on prevalent CNNs across multiple benchmarks.

Conclusions:

• MLPruner offers an effective and autonomous solution for filter pruning in CNNs.
• The mask learning approach preserves the integrity of the weight training process.
• This method contributes to more efficient and practical deployment of deep learning models.