ReCNAS: Resource-Constrained Neural Architecture Search Based on Differentiable Annealing and Dynamic Pruning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Differentiable Neural Architecture Search (NAS) offers efficiency but faces challenges like model collapse and poor resource constraint adherence.
• Existing NAS methods often yield suboptimal architectures that do not meet specific hardware limitations (e.g., FLOPs, latency).

Purpose of the Study:

• To propose a novel resource-constrained NAS (ReCNAS) method for efficient searching of high-performance architectures.
• To address limitations of existing differentiable NAS methods, including model collapse, search-evaluation correlation issues, and inefficient hardware deployment.

Main Methods:

• Introduced an elastic densely connected layerwise search space decoupling depth from operations.
• Implemented group annealing and progressive pruning to enhance efficiency and bridge the search-evaluation gap.
• Developed a resource-constrained architecture generation method using dynamic programming for channel pruning and scalability.

Main Results:

• ReCNAS demonstrated efficiency and search stability in discovering high-performance architectures across various datasets and tasks.
• Searched architectures met target resource constraints precisely without tuning, outperforming other NAS methods.
• The proposed method showed strong generalizability for complex vision tasks.

Conclusions:

• ReCNAS effectively overcomes key limitations in differentiable NAS, enabling efficient and accurate architecture search under resource constraints.
• The method produces scalable, high-performance architectures suitable for diverse computational requirements and complex vision applications.