Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) possess inherent redundancy, allowing for filter/channel removal.
• Current CNN training objectives neglect redundancy, leading to random distribution and potential accuracy drops upon pruning.
• Existing methods often require extensive fine-tuning after pruning to recover performance.

Purpose of the Study:

• To propose a novel training method that manipulates redundancy for effective network pruning.
• To develop a technique that facilitates the removal of redundant filters/channels without compromising network performance.
• To enhance the efficiency and accuracy of CNNs through organized redundancy.

Main Methods:

• Introduction of a novel optimization algorithm: centripetal Stochastic Gradient Descent (C-SGD).
• C-SGD intentionally creates identical filters during training, establishing ideal redundancy patterns.
• Simultaneous application of C-SGD across all layers, including very deep CNNs.

Main Results:

• C-SGD demonstrates superior performance on CIFAR and ImageNet datasets compared to existing methods.
• Organized redundancy achieved by C-SGD leads to better network efficiency and accuracy.
• The method enables pruning without the need for post-training fine-tuning.

Conclusions:

• C-SGD effectively organizes redundancy in CNNs, simplifying network pruning.
• The proposed method offers an efficient and accurate approach to CNN optimization and compression.
• C-SGD presents a promising direction for developing more streamlined and performant deep learning models.