Evolutionary neural architecture search combining multi-branch ConvNet and improved transformer.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Manual design of deep convolutional neural networks (CNNs) is challenging due to complex topologies.
• Existing neural architecture search (NAS) methods often face high computational costs or design inflexibility.

Purpose of the Study:

• To propose an efficient and flexible automated method for designing CNN architectures.
• To address the limitations of current NAS techniques.

Main Methods:

• An evolutionary neural architecture search (ENAS) framework is developed, integrating an improved Transformer with a multi-branch Convolutional Neural Network (ConvNet).
• A novel 'batch-free normalization Transformer Block' (BFNTBlock) is introduced, combining batch-free normalization (BFN) and batch normalization (BN) to improve feature representation and mitigate estimation shift.
• The multi-branch block enhances feature space richness and network representational capacity.

Main Results:

• The proposed ENAS method achieved high accuracies of 97.24% on CIFAR10 and 80.06% on CIFAR100.
• The method demonstrated high computational efficiency, requiring only 1.46 and 1.53 GPU days for training on CIFAR10 and CIFAR100, respectively.
• Superior performance was observed on real-world datasets (GTSRB and NEU-CLS) compared to common methods, validating its universality.

Conclusions:

• The proposed ENAS method effectively automates the design of high-performance and computationally efficient deep neural networks.
• The BFNTBlock design proves beneficial for leveraging both local and long-range dependencies while improving model stability.
• The method shows strong generalization capabilities across diverse computer vision tasks.