Neural Architecture Search Using Covariance Matrix Adaptation Evolution Strategy.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Evolution-based neural architecture search (NAS) is computationally expensive due to training each architecture from scratch.
• Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is effective for hyperparameter tuning but underutilized in NAS.

Purpose of the Study:

• To introduce CMANAS, a novel framework applying CMA-ES to deep neural architecture search.
• To significantly reduce the search time and computational cost associated with NAS.

Main Methods:

• Utilizing the faster convergence of CMA-ES for NAS.
• Employing a trained one-shot model (OSM) to predict architecture fitness, avoiding separate training.
• Implementing an architecture-fitness table (AF table) to store and reuse evaluated architectures.
• Modeling architectures with a normal distribution updated by CMA-ES.

Main Results:

• CMANAS demonstrates superior performance compared to existing evolution-based NAS methods.
• Significant reduction in search time was achieved across various datasets and search spaces.
• Effectiveness validated on CIFAR-10, CIFAR-100, ImageNet, and ImageNet16-120 datasets.

Conclusions:

• CMANAS presents a viable and efficient alternative for evolution-based NAS.
• The study successfully extends the application of CMA-ES to the domain of deep neural architecture search.