Surrogate-Assisted Particle Swarm Optimization for Evolving Variable-Length Transferable Blocks for Image...

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) excel at image classification but manual design is complex.
• Neural architecture search (NAS) automates CNN design but incurs high computational costs.

Purpose of the Study:

• To develop an effective and efficient NAS method using surrogate-assisted particle swarm optimization (PSO).
• To reduce the computational burden associated with designing complex CNN architectures.

Main Methods:

• Proposed a novel surrogate model and a new surrogate dataset creation method.
• Developed a new encoding strategy for variable-length CNN blocks.
• Integrated these components into a PSO algorithm for automated CNN evolution.

Main Results:

• Achieved competitive error rates: 3.49% on CIFAR-10, 18.49% on CIFAR-100, and 1.82% on SVHN.
• Reduced training to 3 GPU-days by avoiding 80.1% of CNN block training via surrogate acceleration.
• Demonstrated transferability of learned CNN blocks across datasets (CIFAR-100, SVHN, ImageNet).

Conclusions:

• The proposed surrogate-assisted PSO method efficiently automates CNN architecture search.
• This approach significantly lowers computational requirements for NAS.
• Learned CNN blocks exhibit strong transferability, enabling broader application.