Graph Embedding Comparator for Evolutionary Neural Architecture Search with Isomorphic Multi-Comparison

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning
• Computer Science

Background:

• Designing effective neural network architectures is a critical challenge in deep learning.
• Automated Neural Architecture Search (NAS) methods often rely on inadequate descriptors or predictors, limiting their ability to capture network complexity and provide reliable guidance.
• Existing NAS approaches struggle with the structural complexity of candidate networks, leading to suboptimal search outcomes.

Purpose of the Study:

• To introduce a novel evolutionary NAS framework, Graph Embedding Comparator with Isomorphic Multi-Comparison (GEC-IMC).
• To develop a method for learning architecture representations directly from their graph structure.
• To enhance the robustness and efficiency of NAS by improving performance prediction accuracy.

Main Methods:

• Utilized a graph convolutional network to encode neural architectures into embeddings.
• Employed a contrastive learning strategy to map architectures with similar accuracy closer in the embedding space.
• Developed a comparator for precise pairwise performance estimation and incorporated an isomorphic multi-comparison mechanism for robust ranking.

Main Results:

• GEC-IMC achieved state-of-the-art performance on standard NAS benchmarks.
• The framework demonstrated improved robustness compared to existing performance predictors.
• Ablation studies confirmed the effectiveness of embedding learning and multi-comparison in boosting search efficiency.

Conclusions:

• GEC-IMC offers a more effective approach to NAS by leveraging graph structure representations.
• The combination of learned embeddings and multi-comparison significantly enhances search robustness and efficiency.
• This framework represents a significant advancement in automating the design of complex neural architectures.