Multi-objective evolutionary optimization for hardware-aware neural network pruning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Engineering

Background:

• Neural network pruning reduces computational complexity but conventional methods use inadequate metrics.
• Hardware-aware network pruning is gaining traction due to new hardware and limitations of existing pruning techniques.
• Balancing network accuracy and hardware efficiency (latency, memory) presents a multi-objective challenge.

Purpose of the Study:

• To address the multi-objective nature of hardware-aware network pruning.
• To propose and evaluate a novel approach using Multi-Objective Evolutionary Algorithms (MOEAs).
• To introduce a specific memetic MOEA, HAMP, for efficient hardware-aware network pruning.

Main Methods:

• Formulating hardware-aware network pruning as a multi-objective optimization problem.
• Developing a novel memetic MOEA named HAMP.
• HAMP integrates an efficient portfolio-based selection mechanism and a surrogate-assisted local search.

Main Results:

• MOEAs can provide a set of Pareto-optimal solutions for hardware-aware network pruning.
• The proposed HAMP algorithm demonstrates superior performance compared to state-of-the-art methods.
• Empirical studies validate the effectiveness of MOEAs in solving this complex optimization task.

Conclusions:

• Multi-Objective Evolutionary Algorithms offer a promising framework for hardware-aware neural network pruning.
• The HAMP algorithm effectively balances network accuracy and hardware efficiency.
• This work advances the field by providing a more suitable optimization strategy for efficient deep learning models.