Architecture Augmentation for Performance Predictor via Graph Isomorphism.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Neural Architecture Search (NAS) automates deep neural network (DNN) design.
• NAS is computationally intensive due to extensive DNN training.
• Performance predictors reduce NAS costs but require many trained architectures.

Purpose of the Study:

• To address the challenge of acquiring sufficient training data for performance predictors in NAS.
• To propose an effective DNN architecture augmentation method to enhance performance predictors.
• To reduce the computational cost associated with Neural Architecture Search.

Main Methods:

• Introduced a graph isomorphism-based architecture augmentation method (GIAug).
• Developed a mechanism to generate diverse annotated architectures efficiently using graph isomorphism.
• Designed a generic encoding method for DNN architectures compatible with prediction models.

Main Results:

• GIAug significantly enhances the performance of state-of-the-art predictors.
• Experiments on CIFAR-10 and ImageNet demonstrate GIAug's effectiveness across various search spaces.
• Achieved up to three orders of magnitude reduction in computation cost on ImageNet with comparable performance.

Conclusions:

• GIAug effectively augments DNN architectures, improving performance predictors for NAS.
• The method offers a significant reduction in computational expense for NAS.
• GIAug is a flexible tool applicable to various performance predictor-based NAS algorithms.