Cluster-Based Structural Redundancy Identification for Neural Network Compression

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Large neural networks pose deployment challenges on resource-constrained edge devices.
• Network pruning is a key technique for model compression, with current methods focusing on filter importance.
• Existing importance-based pruning may overlook structural redundancy and functional similarity among filters.

Purpose of the Study:

• To propose a novel model pruning framework that addresses structural redundancy in neural networks.
• To identify and remove functionally similar filters, rather than just unimportant ones.
• To develop an automated pruning scheme for determining layer-wise pruning rates.

Main Methods:

• Utilizing clustering analysis on neural network filters within each layer to group similar filters.
• Developing a criterion to identify redundant filters within identified clusters of similar filters.
• Implementing an automated scheme to dynamically set the pruning rate for each layer.

Main Results:

• The proposed clustering-based redundancy identification framework effectively compresses neural network models.
• Experiments show superior performance compared to traditional importance-based pruning methods.
• The framework demonstrates effectiveness across various benchmark network architectures and datasets.

Conclusions:

• Identifying and pruning functionally similar filters is a more effective strategy than solely relying on importance.
• The proposed framework offers an automated and efficient approach to model compression for edge deployment.
• This method advances the field of neural network model compression by focusing on structural redundancy.