Neural architecture search using network embedding and generative adversarial networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Surrogate models accelerate neural architecture search (NAS) by predicting performance, avoiding costly full training.
• Collecting sufficient labeled data for surrogate models in NAS is a significant bottleneck.
• Existing NAS methods struggle with limited data for effective surrogate model training.

Purpose of the Study:

• To propose GNE-NAS, a surrogate-assisted swarm optimization algorithm with network embedding for efficient NAS.
• To enhance surrogate model performance using generative adversarial networks for data augmentation (GNE-NAS).
• To address the challenge of limited training data for surrogate models in NAS.

Main Methods:

• Employed unsupervised learning for meaningful representation of neural architectures in an embedding space.
• Utilized generative adversarial networks (GANs) for data augmentation to improve surrogate model robustness.
• Integrated network embedding and GAN-based data augmentation within a swarm optimization framework for NAS.

Main Results:

• Network embedding positions structurally similar architectures closer in the embedding space, aiding surrogate model training.
• GAN-based data augmentation enhances surrogate model robustness and reduces the need for extensive real evaluations.
• Experimental results on NASBench search spaces show comparable or superior performance of surrogate models with network embedding.

Conclusions:

• GNE-NAS significantly improves surrogate model performance in NAS, especially with limited data.
• The proposed method outperforms state-of-the-art neural architecture search algorithms.
• Network embedding and data augmentation are effective strategies for enhancing surrogate-assisted NAS.