Embedding channel pruning within the CNN architecture design using a bi-level evolutionary approach.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep neural networks, particularly Convolutional Neural Networks (CNNs), have driven advancements in machine learning and computer vision.
• Hyperparameter selection in CNNs is critical due to exponentially growing search spaces with increased layers.
• Existing pruning algorithms require pre-trained architectures, neglecting pruning during the design phase.

Purpose of the Study:

• To address the challenge of optimizing CNN architecture design and channel pruning simultaneously.
• To develop a bi-level optimization framework that integrates architecture generation and pruning processes.
• To enhance the efficiency and accuracy of CNNs through a unified design and pruning strategy.

Main Methods:

• A bi-level optimization approach was developed, with the upper level for architecture generation and the lower level for channel pruning.
• A co-evolutionary migration-based algorithm was employed as the search engine for the bi-level optimization.
• The proposed method, CNN-D-P (bi-level CNN design and pruning), was evaluated on CIFAR-10, CIFAR-100, and ImageNet datasets.

Main Results:

• The CNN-D-P method demonstrated effectiveness in optimizing CNN architectures for image classification.
• Comparison tests against state-of-the-art architectures validated the proposed technique's performance.
• The bi-level approach showed potential in transforming medium-quality architectures into highly efficient and accurate models.

Conclusions:

• The integrated bi-level optimization of CNN design and pruning is a promising direction for developing efficient deep learning models.
• CNN-D-P offers a novel solution for the critical challenge of hyperparameter optimization and network pruning.
• The method's validation on benchmark datasets suggests its applicability to real-world computer vision tasks.