Filter Pruning by High-Order Spectral Clustering.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) often contain significant redundancy, leading to large model sizes.
• Existing filter pruning methods primarily rely on distance metrics, which fail to capture complex correlations and are unsuitable for high-dimensional features.
• This limitation hinders effective compression of deep learning models.

Purpose of the Study:

• To develop an advanced filter pruning strategy for CNNs that addresses the limitations of distance-based methods.
• To improve the accuracy and efficiency of model compression by identifying and removing redundant filters more effectively.
• To achieve substantial model size reduction with minimal or no loss in performance.

Main Methods:

• Proposes a novel pruning strategy based on high-order spectral clustering.
• Utilizes a hypergraph structure to model complex correlations among filters.
• Employs hypergraph structure learning to extract high-order information for filter clustering and redundancy identification.

Main Results:

• The proposed method demonstrates superior performance compared to state-of-the-art techniques across various CNN models and datasets.
• Achieved a 57.1% reduction in Floating Point Operations (FLOPs) for ResNet50 on ImageNet without any accuracy drop.
• Represents a breakthrough in lossless pruning with a high compression ratio.

Conclusions:

• High-order spectral clustering provides a more effective approach for identifying and removing redundant filters in CNNs.
• The proposed hypergraph-based method enables significant model compression while preserving accuracy.
• This work sets a new benchmark for lossless pruning in deep learning models.