Compact Neural Network via Stacking Hybrid Units.

Area of Science:

• Deep learning
• Computer vision
• Artificial intelligence

Background:

• Pruning techniques are vital for reducing parameters in deep neural networks (NNs).
• Unstructured pruning leads to sparse, irregular weights, while structured pruning requires complex criteria.
• Existing methods face limitations in efficiency and complexity for network compression.

Purpose of the Study:

• Introduce BUnit-Net, a new method for constructing compact neural networks.
• To develop a pruning technique that avoids complex criteria and maintains regular weight tensors.
• To offer an effective alternative for network compression and model optimization.

Main Methods:

• BUnit-Net constructs compact NNs by systematically stacking designed basic units.
• The independence among units allows for fewer weight parameters and dense weight tensors.
• The method was evaluated on diverse backbones and benchmark datasets, compared against state-of-the-art pruning methods.

Main Results:

• BUnit-Net achieves comparable classification accuracy to existing methods.
• Demonstrates significant reduction in computational resources, saving approximately 80% FLOPs and 73% parameters.
• Proposed two new metrics to effectively evaluate compression performance trade-offs.

Conclusions:

• Stacking basic units offers a promising and effective new direction for neural network compression.
• BUnit-Net provides a simpler yet powerful alternative to traditional pruning methods.
• The approach successfully balances compression efficiency with model performance.