An Efficient Multi-Objective Evolutionary Zero-Shot Neural Architecture Search Framework for Image Classification.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Neural Architecture Search (NAS) automates network design but relies on costly validation performance on large datasets.
• For small datasets, validation performance inaccurately estimates test performance, leading to suboptimal NAS results.
• Existing NAS methods struggle with efficiency and accuracy on limited data.

Purpose of the Study:

• To develop an efficient, multi-objective, evolutionary, zero-shot NAS framework.
• To address the limitations of traditional NAS evaluation methods on small-scale datasets.
• To improve the accuracy and throughput of NAS by employing training-free architecture evaluation.

Main Methods:

• Proposed an efficient multi-objective evolutionary zero-shot NAS framework.
• Introduced a general principle for designing zero-cost metrics, unifying existing ones and creating new ones.
• Developed an efficient computational method for multi-zero-cost metrics using a single forward-backward pass.

Main Results:

• The proposed zero-shot NAS framework demonstrated accurate and high-throughput performance on MedMNIST.
• Achieved 20x speedup compared to previous state-of-the-art methods.
• Validated the effectiveness of zero-cost metrics for NAS on NAS-Bench-201 and MedMNIST.

Conclusions:

• Zero-cost metrics provide a viable and efficient alternative for architecture evaluation in NAS, particularly for small datasets.
• The proposed framework significantly enhances NAS efficiency and accuracy without requiring extensive training.
• This approach offers a promising direction for resource-constrained NAS applications.