Subarchitecture Ensemble Pruning in Neural Architecture Search.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Neural Architecture Search (NAS) offers flexibility in neural network design but incurs substantial computational costs.
• Existing ensemble methods for NAS reduce costs but often overlook diversity, leading to redundant subarchitectures.

Purpose of the Study:

• To propose a novel pruning method for NAS ensembles, termed subarchitecture ensemble pruning in neural architecture search (SAEP).
• To enhance diversity within NAS ensembles and achieve smaller, efficient subensemble architectures with maintained performance.

Main Methods:

• Developed SAEP, a pruning technique specifically designed for NAS ensembles.
• Investigated three distinct strategies for selecting and pruning subarchitectures during the search process.

Main Results:

• Experimental results demonstrate SAEP's effectiveness in significantly reducing the number of subarchitectures.
• The proposed method achieves comparable performance to unpruned ensembles, validating its efficiency.

Conclusions:

• SAEP successfully addresses the redundancy issue in NAS ensembles by leveraging diversity.
• This method offers a computationally efficient approach to designing high-performing neural architectures.