Lossless Reconstruction of Convolutional Neural Network for Channel-Based Network Pruning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Network pruning is crucial for reducing computational costs and parameters in convolutional neural networks (CNNs).
• Existing pruning methods often neglect the reconstruction phase, limiting practical deployment on devices.
• Efficient deployment of pruned CNNs requires careful consideration of network components post-pruning.

Purpose of the Study:

• To propose a novel reconstruction process for channel-based network pruning.
• To enable lossless reconstruction of pruned networks for practical device application.
• To improve the performance and efficiency of pruned CNNs on downstream tasks.

Main Methods:

• Developed a reconstruction process focusing on residual blocks, skip connections, and convolution layers.
• Applied union operation and index alignment for reconstructing residual blocks and skip connections.
• Incorporated batch normalization into the reconstruction of compressed convolution layers.

Main Results:

• Achieved higher accuracy in image classification (1.93%), semantic segmentation (2.2 mIoU), and object detection (0.054 mAP) compared to small models.
• Demonstrated significant latency reduction: 8.15× on Raspberry Pi and 5.29× on Jetson Nano.
• Validated the method's effectiveness across various downstream tasks.

Conclusions:

• The proposed reconstruction method effectively enhances pruned CNN performance for real-world applications.
• This approach bridges the gap between network pruning research and practical device deployment.
• The method offers a viable solution for deploying efficient and high-performing deep learning models on edge devices.