A Generic Graph-Based Neural Architecture Encoding Scheme With Multifaceted Information.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Neural Architecture Search (NAS) automates the design of high-performing neural networks.
• Traditional NAS methods face significant computational challenges due to large search spaces.
• Predictor-based NAS methods aim to improve efficiency by learning performance predictors.

Purpose of the Study:

• To enhance the sample efficiency and effectiveness of Neural Architecture Search.
• To develop a novel encoding scheme that better represents neural network architectures.
• To improve the generalization ability of performance predictors in NAS.

Main Methods:

• Introduced GATES (Graph-based neural ArchiTecture Encoding Scheme) to model architectures.
• Proposed GATES++, an enhanced encoding scheme incorporating operation-level and architecture-level computing semantics.
• Trained GATES++ by leveraging multifaceted information about neural network computations.

Main Results:

• GATES++ demonstrates improved performance over the original GATES.
• Both operation-level and architecture-level semantic information individually contribute to performance gains.
• GATES++ discovers superior architectures with fewer evaluated samples compared to baseline methods.

Conclusions:

• Incorporating computing semantics into neural architecture encoding significantly boosts NAS efficiency.
• GATES++ offers a more effective approach to designing neural network architectures.
• The proposed method advances the field of automated machine learning by reducing computational overhead.