Population Historical Information-Driven Evolutionary Multitask Neural Architecture Search

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision
• Natural Language Processing

Background:

• Neural Architecture Search (NAS) automates neural network design, with evolutionary NAS being highly successful.
• Multitask NAS (MT-NAS) aims for efficient architecture discovery across diverse domains like computer vision and NLP.
• Existing MT-NAS methods suffer from redundant searches and negative transfer due to poor utilization of historical data and unguided task interactions.

Purpose of the Study:

• To propose a novel evolutionary multitask neural architecture search (MT-NAS) algorithm.
• To address limitations of existing MT-NAS methods, specifically redundant search and negative transfer.
• To enhance the efficiency and effectiveness of NAS through improved information utilization and task interaction.

Main Methods:

• Developed a population historical information-driven evolutionary multitask neural architecture search (HIMT-NAS) algorithm.
• Recorded population historical information (operation and topology) for each generation.
• Systematically utilized historical information to guide evolutionary search directions, preventing redundant exploration.
• Adjusted cross-task knowledge transfer probabilities based on measured task similarity derived from historical information patterns.
• Updated transfer probabilities dynamically when information proved beneficial across multiple tasks.

Main Results:

• The proposed HIMT-NAS algorithm demonstrated consistent advantages over single-task NAS and existing MT-NAS methods.
• Experiments were conducted on benchmark datasets including MedMNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet.
• The method effectively reduced redundant search by leveraging population historical data.
• Improved cross-task knowledge transfer was achieved by dynamically adjusting probabilities based on task similarity.

Conclusions:

• The HIMT-NAS algorithm offers a significant advancement in efficient and effective multitask neural architecture search.
• Leveraging population historical information is crucial for optimizing NAS efficiency and mitigating negative transfer.
• The proposed method provides a robust framework for automated neural network design across various AI domains.