Split-Level Evolutionary Neural Architecture Search With Elite Weight Inheritance.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Neural Architecture Search (NAS) automates deep model construction.
• Evolutionary Computation (EC) offers gradient-free search but struggles with discrete filter numbers.
• Current EC-NAS methods are inefficient due to extensive performance evaluations.

Purpose of the Study:

• To develop a more flexible and efficient EC-based NAS method.
• To address the limitations of discrete search spaces for filter numbers.
• To reduce the computational cost of performance evaluation in NAS.

Main Methods:

• Proposes a split-level Particle Swarm Optimization (PSO) approach for NAS.
• Each particle dimension is split into integer and fractional parts for flexible layer configuration and filter number search.
• Introduces elite weight inheritance with an online updating weight pool to save evaluation time.
• Develops a multi-objective customized fitness function to control architecture complexity.

Main Results:

• The proposed split-level evolutionary NAS (SLE-NAS) method demonstrates computational efficiency.
• SLE-NAS outperforms state-of-the-art competitors on image classification benchmarks.
• Achieves superior performance at significantly lower complexity compared to existing methods.

Conclusions:

• SLE-NAS offers a flexible and efficient solution for Neural Architecture Search.
• The split-level PSO approach effectively handles filter number optimization.
• The method significantly reduces computational overhead while maintaining high performance.