Dynamical Conventional Neural Network Channel Pruning by Genetic Wavelet Channel Search for Image Classification

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) face high computational costs, limiting their deployment on resource-constrained devices.
• Existing network pruning methods often use fixed pruning ratios, neglecting layer-specific channel variations and leading to suboptimal performance.
• The need for dynamic and adaptive pruning strategies is crucial for efficient DNN compression.

Purpose of the Study:

• To propose a novel genetic wavelet channel search (GWCS) framework for efficient neural network pruning.
• To dynamically identify and prune the most representative and discriminative channels within different convolutional layers.
• To improve both accuracy and compression rates of deep convolutional neural networks (CNNs).

Main Methods:

• Modeled the pruning process as a multi-stage genetic optimization procedure.
• Encoded all channels and divided them into search spaces based on functional convolutional layers.
• Developed a wavelet channel aggregation-based fitness function for dynamic channel selection and pruning.

Main Results:

• Evaluated GWCS on CIFAR-10, CIFAR-100, and ImageNet datasets using ResNet and VGGNet architectures.
• Demonstrated superior performance compared to state-of-the-art pruning algorithms in terms of accuracy and compression rate.
• Achieved over 73.1% reduction in FLOPs for ResNet-32 on CIFAR-100 with a 0.79% accuracy improvement.

Conclusions:

• The proposed GWCS framework effectively addresses the limitations of conventional pruning methods.
• GWCS offers a dynamic and adaptive approach to neural network pruning, optimizing channel selection.
• The method significantly enhances model efficiency and performance, making DNNs more suitable for resource-limited environments.