Sample-Efficient Neural Architecture Search by Learning Actions for Monte Carlo Tree Search.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Neural Architecture Search (NAS) automates neural network design.
• Current MCTS-based NAS methods use manually designed action spaces, leading to inefficient exploration.
• This inefficiency hinders optimizing performance metrics like accuracy.

Purpose of the Study:

• To introduce Latent Action Neural Architecture Search (LaNAS) for improved sample efficiency in NAS.
• To develop a method that learns actions to guide the search towards high-performing architectures.
• To address the limitations of manually designed action spaces in MCTS-based NAS.

Main Methods:

• LaNAS learns actions to recursively partition the architecture search space.
• It categorizes regions as 'good' or 'bad' based on network performance.
• The search process is biased towards 'good' regions for efficiency.

Main Results:

• LaNAS demonstrated at least an order of magnitude improvement in sample efficiency across three NAS tasks.
• It outperformed evolutionary algorithms, Bayesian optimization, and random search.
• Achieved 99.0% accuracy on CIFAR-10 and 80.8% top-1 on ImageNet (600 MFLOPS) with only 800 samples.

Conclusions:

• LaNAS significantly enhances sample efficiency in Neural Architecture Search.
• The learned latent actions effectively guide the search towards optimal neural network architectures.
• LaNAS offers a superior alternative to existing NAS methods, achieving state-of-the-art results with reduced sample complexity.