ModuleNet: Knowledge-Inherited Neural Architecture Search.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Neural Architecture Search (NAS) often overlooks valuable knowledge from pre-existing deep learning models.
• The computationally intensive nature of NAS necessitates the reuse of existing knowledge rather than starting from scratch.

Purpose of the Study:

• To investigate the types of knowledge within existing models that can be leveraged for new architecture designs.
• To propose ModuleNet, a novel NAS algorithm designed to fully inherit knowledge from existing convolutional neural networks.

Main Methods:

• Decomposing existing convolutional neural networks into modules that retain their weights, forming a knowledge base.
• Sampling and searching for new architectures using this knowledge base.
• Employing the NSGA-II algorithm for direct macro-space architecture search without module parameter tuning.

Main Results:

• ModuleNet efficiently evaluates new architectures without requiring weight tuning in convolutional layers.
• Inherited knowledge enables the search to achieve superior performance compared to original architectures across datasets like CIFAR10, CIFAR100, and ImageNet.

Conclusions:

• ModuleNet offers an efficient and effective approach to neural architecture search by leveraging knowledge from existing models.
• The proposed method demonstrates the potential of knowledge inheritance in significantly improving deep learning model performance and search efficiency.