On the compression of neural networks using ℓ₀-norm regularization and weight pruning

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Implementation complexity and high costs hinder neural network deployment, especially for edge intelligence and embedded systems.
• Network compression techniques are crucial for reducing deployment costs while preserving inference accuracy.

Purpose of the Study:

• To develop a novel neural network compression scheme.
• To address the challenges of deploying complex neural networks in resource-constrained environments.

Main Methods:

• Developed a novel L0-norm-based regularization to induce network sparseness during training.
• Applied pruning techniques to remove smaller weights from the trained network.
• Incorporated L2-norm regularization to prevent overfitting and fine-tuning to enhance performance.

Main Results:

• The proposed compression scheme successfully created smaller, highly effective neural networks.
• Experimental results demonstrated the effectiveness of the novel compression approach.
• Comparisons with competing methods highlighted the advantages of the proposed scheme.

Conclusions:

• The novel compression scheme offers a viable solution for deploying efficient neural networks on edge devices.
• The method balances network size reduction with the maintenance of satisfactory inference accuracy.
• This research contributes to the advancement of efficient deep learning models for practical applications.