Adaptive Channel Allocation for Robust Differentiable Architecture Search.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Differentiable Architecture Search (DARTS) offers efficiency but suffers from skip connection accumulation, leading to instability and reduced robustness in deep networks.
• Existing methods to control skip connections rely on manual design or hyperparameters, limiting their generalizability.

Purpose of the Study:

• To propose a novel, implicit method for managing skip connections in DARTS that avoids human priors and hyperparameters.
• To enhance the stability and robustness of neural network architectures discovered by DARTS, particularly with increased training epochs.

Main Methods:

• Introduced an Adaptive Channel Allocation (ACA) strategy that implicitly searches for skip connections.
• Implemented channel importance ranking and allocation during the evaluation stage to refill unimportant channels.
• Designed ACA as a general-purpose approach compatible with various DARTS variants.

Main Results:

• ACA effectively addresses the skip connection accumulation and collapse issues in DARTS.
• Experiments demonstrate ACA's superior performance in improving robustness compared to existing methods.
• The strategy proved effective across diverse datasets and DARTS variants.

Conclusions:

• The ACA strategy offers a robust and generalizable solution for enhancing DARTS.
• Implicitly managing skip connections through channel allocation is a more effective approach than explicit control.
• ACA significantly improves the practical applicability of DARTS by enhancing stability and robustness.