PCA driven mixed filter pruning for efficient convNets.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) face deployment challenges on resource-limited devices due to high memory and computational demands.
• Existing pruning techniques often leave residual redundancy in DNN models, limiting compression efficiency.

Purpose of the Study:

• To develop an effective mixed filter pruning strategy for compressing DNNs.
• To enable efficient deployment of DNNs on devices with constrained resources.

Main Methods:

• A two-stage pruning approach combining Principal Component Analysis (PCA) and geometric median.
• PCA identifies key filters for initial network reconstruction and reduction.
• Geometric median further identifies and removes redundant filters for enhanced compression.

Main Results:

• Significant reductions in operations and parameters across various datasets (CIFAR-10, CIFAR-100, ILSVRC 2017) and models (VGG-16, AlexNet, VGG-19).
• Achieved compression rates up to 18.56x for operations and 36x for parameters.
• Maintained high accuracy, with losses as low as 0.5% or even accuracy gains (1.2% for AlexNet).

Conclusions:

• The proposed mixed pruning approach is effective for compressing DNNs.
• The method enables substantial model compression while preserving or improving accuracy.
• This technique facilitates the deployment of DNNs on resource-constrained edge devices.