EDP: An Efficient Decomposition and Pruning Scheme for Convolutional Neural Network Compression.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Deep neural networks (DNNs) are computationally intensive, limiting their real-world application.
• Existing model compression techniques involve complex, multi-stage processes and rely on potentially unsuitable assumptions.

Purpose of the Study:

• To develop an efficient and simplified model compression scheme for DNNs.
• To address limitations of current methods, including cumbersome training and restrictive assumptions.

Main Methods:

• Proposed an efficient decomposition and pruning (EDP) scheme using a compressed-aware block.
• Decomposed network layers into weight and coefficient matrices, applying regularizers to achieve low-rank and sparsity.
• Introduced a Pruning & Merging (PM) module for further compression and optimization.

Main Results:

• The EDP scheme achieved simultaneous low-rank decomposition and channel pruning in a single training stage.
• Experimental results demonstrated high compression ratios with minimal accuracy loss.
• Outperformed 17 competitors in compression rate, accuracy, inference time, and runtime memory.

Conclusions:

• The proposed EDP scheme offers an efficient and effective solution for DNN model compression.
• This method simplifies the compression process while maintaining high performance.
• EDP represents a significant advancement in making DNNs more practical for real-world applications.