A Fast Evolutionary Knowledge Transfer Search for Multiscale Deep Neural Architecture.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Neural Architecture Search (NAS) automates neural network design, but high computational costs hinder its development.
• Evolutionary NAS (ENAS) offers a robust approach but still faces efficiency challenges.

Purpose of the Study:

• To present a fast ENAS framework to reduce computational costs and improve efficiency.
• To investigate a novel framework for multiscale convolutional networks.

Main Methods:

• Developed an Evolutionary Knowledge Transfer Search (EKTS) framework combining global and local optimization.
• Utilized evolutionary computation for robust global search of neural architectures.
• Integrated knowledge transfer and local fast learning to accelerate the search process.
• Explored a multiscale gray-box structure combining Bandelet transform with convolution.

Main Results:

• The proposed EKTS framework significantly reduces computational costs associated with NAS.
• The framework effectively searches and designs multiscale convolutional network architectures.
• The developed architectures demonstrated superior performance compared to over 40 existing models.

Conclusions:

• The fast ENAS framework based on EKTS offers an efficient and effective solution for designing advanced neural networks.
• The novel multiscale gray-box structure enhances network approximation, learning, and generalization capabilities.