PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Multiscale features are crucial for modern convolutional neural networks (CNNs), improving performance across various vision tasks.
• Existing plug-and-play blocks enhance multiscale representation but are often complex and suboptimal due to manual design.

Purpose of the Study:

• To introduce PP-NAS, a novel approach for automatically designing efficient and effective plug-and-play blocks using neural architecture search (NAS).
• To overcome the limitations of manual design in creating complex CNN blocks for superior multiscale feature representation.

Main Methods:

• Developed a new search space named PPConv tailored for plug-and-play blocks.
• Implemented a NAS algorithm incorporating one-level optimization, zero-one loss, and connection existence loss.
• Minimized the optimization gap between super-networks and sub-architectures to enable efficient block discovery.

Main Results:

• PP-NAS demonstrates superior performance on image classification, object detection, and semantic segmentation tasks.
• The automatically designed blocks achieve competitive results even without extensive retraining.
• PP-NAS outperforms state-of-the-art CNN architectures like ResNet, ResNeXt, and Res2Net.

Conclusions:

• PP-NAS offers an effective and automated solution for designing advanced plug-and-play blocks for CNNs.
• The proposed method significantly enhances multiscale representation capabilities, leading to improved performance in diverse computer vision applications.
• The availability of the code facilitates further research and application of PP-NAS.