Toward Less Constrained Macro-Neural Architecture Search.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Neural Architecture Search (NAS) methods often rely on human-defined constraints, limiting the exploration of novel network architectures.
• Existing NAS approaches typically focus on cell-based search spaces, restricting the design of entire network structures (macro-search).

Purpose of the Study:

• To introduce Less Constrained Macro-Neural Architecture Search (LCMNAS) for broader architectural exploration.
• To develop a NAS method that performs macro-search without predefined heuristics or bounded search spaces.
• To achieve state-of-the-art performance with reduced computational cost.

Main Methods:

• LCMNAS utilizes weighted directed graphs (WDGs) to autonomously generate complex, less constrained search spaces informed by existing architectures.
• An evolutionary search strategy generates complete architectures from scratch.
• A mixed-performance estimation approach combines initial architecture information with low-fidelity estimates to predict performance.

Main Results:

• LCMNAS successfully generated both cell-based and macro-based architectures across 14 diverse datasets.
• The method achieved state-of-the-art results with minimal GPU computation.
• Extensive studies validated the effectiveness of LCMNAS components in both cell and macro search settings.

Conclusions:

• LCMNAS significantly advances NAS by enabling unconstrained macro-search, leading to novel and high-performing neural network architectures.
• The proposed method offers an efficient and automated approach to discovering complex architectures, outperforming human-designed networks.