NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Neural Architecture Search (NAS) aims to automate the design of neural networks.
• Existing methods like Bayesian Optimization (BO) and Evolutionary Algorithms (EA) for NAS are computationally expensive and inefficient.
• There is a need for improved NAS strategies that balance performance and search cost.

Purpose of the Study:

• To propose a novel Neural Predictor guided EA for NAS (NPENAS) to enhance exploration ability.
• To introduce two types of neural predictors: a BO acquisition function and a direct performance predictor.
• To develop a new random architecture sampling method to improve upon existing techniques.

Main Methods:

• Developed NPENAS, integrating neural predictors with EA for NAS.
• Designed two neural predictors: NPENAS-BO (using graph-based uncertainty estimation) and NPENAS-NP (direct performance prediction).
• Implemented a novel random architecture sampling strategy.

Main Results:

• NPENAS-BO and NPENAS-NP demonstrated superior performance compared to existing NAS algorithms across five search spaces.
• NPENAS-NP achieved state-of-the-art results on four out of the five NAS search spaces evaluated.
• The proposed random sampling method effectively addressed limitations of prior approaches.

Conclusions:

• NPENAS offers a more efficient and effective approach to neural architecture search.
• The neural predictor guided EA significantly enhances exploration and performance in NAS.
• NPENAS-NP represents a promising direction for achieving state-of-the-art results in NAS.