Mitigating Search Interference With Task-Aware Nested Search.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Neural Architecture Search (NAS) is key to AutoML for efficient model design.
• Weight-sharing in NAS supernets reduces search cost but causes negative interference, especially in multi-task learning.

Purpose of the Study:

• To propose a task-aware nested search method to mitigate interference in multi-task NAS.
• To develop a search-in-search approach for generating task-specific search spaces and architectures.

Main Methods:

• A novel task-aware nested search framework with two phases: space-search and architecture-search.
• The space-search phase discovers optimal, task-specific subspaces using a search space generator.
• The architecture-search phase finds promising architectures within these subspaces, enabling adaptive weight sharing.

Main Results:

• The proposed method significantly reduces search interference in multi-task NAS.
• Achieved superior performance across vision benchmarks (CityScapes, NYUv2, Tiny-Taskonomy).
• Demonstrated improvements in task accuracy, reduced model parameters, and lower latency compared to existing methods.

Conclusions:

• The task-aware nested search effectively addresses negative interference in multi-task NAS.
• This approach offers a more efficient and accurate way to design neural network architectures for diverse tasks.