Kernel-wise difference minimization for convolutional neural network compression in metaverse.

Area of Science:

• Computer Vision
• Deep Learning
• Data Compression

Background:

• Convolutional neural networks (CNNs) excel in computer vision but suffer from increasing complexity, demanding substantial memory and computational resources.
• Model compression is crucial for deploying efficient deep neural networks (DNNs) in resource-constrained environments.

Purpose of the Study:

• To develop a novel algorithm for effective model compression of deep neural networks.
• To address the challenges of large model sizes and high computational costs in CNNs.

Main Methods:

• Formulated a compression problem based on filter-wise difference minimization, inspired by Huffman coding for low-entropy data.
• Proposed a novel algorithm involving filter-level pruning, minimizing inter-filter differences, and filter permutation for enhanced compression.

Main Results:

• Achieved a compression rate of 94× on Lenet-5 and 50× on VGG16.
• Demonstrated significant reduction in deep neural network size while maintaining high accuracy.

Conclusions:

• The proposed method effectively compresses deep neural networks, offering a promising solution for efficient model deployment.
• This research contributes valuable insights into addressing the challenges of model compression in deep learning applications.