Evolutionary Multiobjective Neural Architecture Search for Binary Neural Networks by Two-Stage Optimization

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Binary Neural Networks (BNNs) offer extreme model compression for resource-limited environments.
• Designing efficient BNN architectures is challenging due to specialized binarized operations.
• Neural Architecture Search (NAS) presents a viable solution for high-performance BNN design.

Purpose of the Study:

• To propose a multiobjective evolutionary NAS algorithm (MO-TS-BNAS) for BNNs.
• To address the need for networks with varying parameter sizes and performance levels.
• To optimize BNN architecture design balancing model size and error.

Main Methods:

• Utilized the ApproxSign function for gradient approximation in BNN training.
• Introduced auxiliary objectives in nondominated sorting to mitigate the small model trap.
• Implemented a two-stage training strategy with path dropout and improved mini-batch gradient descent.
• Binarized the full-precision baseline search space for comparative analysis.

Main Results:

• The MO-TS-BNAS algorithm successfully balances model size and error objectives.
• Experimental validation on CIFAR10 and ImageNet datasets demonstrated the method's effectiveness.
• The proposed approach optimizes BNN architectures for diverse performance requirements.

Conclusions:

• MO-TS-BNAS is an effective method for designing high-performance BNN architectures.
• The algorithm addresses key challenges in BNN architecture search, including model size and accuracy trade-offs.
• This work advances the application of BNNs in mobile and resource-constrained settings.