A Knee-Guided Evolutionary Algorithm for Compressing Deep Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) are powerful but resource-intensive due to large parameters and high computational costs.
• Compressing DNNs via parameter pruning is crucial for real-time applications and resource-constrained devices.
• Manual search for optimal pruning parameters is challenging due to model overparameterization and conflicting objectives.

Purpose of the Study:

• To address the challenge of DNN compression by developing an automated method for parameter pruning.
• To formally define filter pruning as a multiobjective optimization problem.
• To propose a novel algorithm that balances parameter reduction and performance maintenance.

Main Methods:

• Filter pruning is formulated as a multiobjective optimization problem.
• A knee-guided evolutionary algorithm (KGEA) is proposed for automated search.
• A minimum Manhattan distance approach guides the search towards optimal trade-off solutions.
• Parameter importance is assessed based on performance loss to identify redundancy.

Main Results:

• The proposed KGEA effectively searches for optimal trade-off solutions between model size and performance.
• The algorithm identifies a 'knee' solution representing a good balance between compression and accuracy.
• A performance-improved model was achieved without fine-tuning.
• Experiments on LeNet and VGG-19 demonstrated superior performance compared to existing methods.

Conclusions:

• KGEA offers an efficient and automated approach to DNN compression.
• The method successfully balances parameter reduction and performance, facilitating deployment on constrained devices.
• The algorithm's ability to find high-quality solutions without fine-tuning highlights its practical utility.