Complex hybrid weighted pruning method for accelerating convolutional neural networks

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) are computationally intensive, necessitating efficient compression and acceleration techniques.
• Current filter pruning methods, such as norm-based and relation-based approaches, often overlook filter diversity and batch normalization layer impacts, potentially degrading performance.

Purpose of the Study:

• To address the limitations of existing filter pruning methods.
• To introduce a novel complex hybrid weighted pruning method that enhances filter pruning effectiveness and robustness.

Main Methods:

• Empirical analysis of norm-based and similarity-based pruning drawbacks.
• Development of a complex hybrid weighted pruning method evaluating filter correlations, norms, and batch normalization parameters.
• Comprehensive pruning experiments on ResNet architectures using ImageNet and CIFAR-10 datasets.

Main Results:

• The proposed hybrid weighted pruning method effectively identifies and removes redundant filters without significant performance degradation.
• Achieved a 53.5% reduction in floating-point operations for ResNet-50 on ImageNet with only a 0.6% performance loss.
• Demonstrated significant efficacy across different ResNet depths and datasets.

Conclusions:

• The complex hybrid weighted pruning method offers a robust and effective solution for compressing CNNs.
• This approach successfully balances network compression with performance preservation, outperforming existing methods.