Efficient tensor decomposition-based filter pruning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Neural network pruning is crucial for reducing computational costs and memory footprints.
• Conventional pruning methods often rely on simplified filter representations, potentially losing information.
• There is a need for efficient pruning techniques that preserve network performance.

Purpose of the Study:

• To introduce CORING (efficient tensor decomposition-based filter pruning), a novel filter pruning methodology for neural networks.
• To leverage tensor decomposition for efficient and effective neural network compression.
• To demonstrate CORING's superiority over existing state-of-the-art pruning methods.

Main Methods:

• CORING utilizes tensor decomposition, specifically Higher-Order Singular Value Decomposition (HOSVD), to approximate filters in their multidimensional form.
• A novel filter similarity metric is introduced, based on HOSVD's low-rank approximation, enhancing efficiency.
• The methodology was tested across diverse neural network architectures and datasets for various computer vision tasks.

Main Results:

• CORING significantly reduces Multiply-Accumulate operations (MACs) and parameters compared to state-of-the-art methods.
• The method consistently improves validation accuracy across image classification, object detection, instance segmentation, and keypoint detection tasks.
• Ablation studies and qualitative results confirm the efficiency of the tensor-based approach and the retention of essential network features.

Conclusions:

• CORING offers an efficient and effective approach to neural network filter pruning using tensor decomposition.
• The method achieves superior compression rates while enhancing model accuracy.
• CORING represents a significant advancement in model optimization for deep learning applications.