PATNAS: A Path-Based Training-Free Neural Architecture Search.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Neural Architecture Search (NAS) is computationally expensive due to the need for extensive network architecture evaluation.
• Zero-cost proxies offer a solution by rapidly estimating final network performance during early training phases.
• Existing proxies have limitations, often neglecting network structure or task specificity.

Purpose of the Study:

• To introduce a novel zero-cost proxy, Skeleton Path Kernel Trace (SPKT), for Neural Architecture Search.
• To address the limitations of current proxies by incorporating network skeleton path structure information.
• To enhance the efficiency and effectiveness of NAS frameworks.

Main Methods:

• Proposed the Skeleton Path Kernel Trace (SPKT) zero-cost proxy, utilizing network skeleton path structure.
• Integrated SPKT into the PATNAS Bayesian optimization framework for NAS.
• Evaluated SPKT's efficacy across diverse datasets and tasks.

Main Results:

• SPKT demonstrated a high correlation with the final performance of evaluated network architectures.
• The proposed SPKT proxy significantly accelerated the NAS search process.
• SPKT proved effective across multiple tasks, overcoming limitations of previous methods.

Conclusions:

• SPKT is an effective zero-cost proxy for Neural Architecture Search, capturing crucial structural information.
• The integration of SPKT within PATNAS substantially improves NAS efficiency.
• This approach offers a promising direction for reducing the computational burden of designing high-performing neural networks.