Evolutionary Multi-Objective One-Shot Filter Pruning for Designing Lightweight Convolutional Neural Network

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) offer high performance but are resource-intensive.
• Complex DNNs face limitations in mobile and embedded applications due to computational and storage demands.
• Neural network pruning is a key technique for developing efficient, lightweight models.

Purpose of the Study:

• To propose an evolutionary multi-objective one-shot filter pruning method for lightweight convolutional neural networks (CNNs).
• To address the resource limitations of DNNs in edge computing environments.
• To balance model compression with accuracy preservation.

Main Methods:

• A one-shot pruning framework performing filter pruning and fine-tuning in a single step.
• Formulating filter pruning as a constrained multi-objective optimization problem (pruning ratio vs. accuracy).
• Employing a non-dominated sorting-based evolutionary multi-objective algorithm to find Pareto-optimal solutions.

Main Results:

• The proposed method achieved significant filter pruning ratios across various models (LeNet, AlexNet).
• Pruning rates exceeded 85%, 82%, 75%, 65%, 91%, and 68% on tested models with minimal accuracy degradation.
• A set of Pareto solutions offered diverse trade-offs between model size and accuracy.

Conclusions:

• The evolutionary multi-objective one-shot filter pruning method effectively creates lightweight CNNs.
• This approach is suitable for deploying deep learning models on resource-constrained devices.
• The method demonstrates a practical solution for balancing model compression and performance.