RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep Neural Networks (DNNs) require optimization for efficient inference.
• Pruning is a key technique for accelerating DNNs by reducing model complexity.
• Existing pruning methods often require access to training data, limiting their applicability.

Purpose of the Study:

• Introduce RED++, a novel data-free pruning protocol for Deep Neural Networks (DNNs).
• Enable inference runtime acceleration without requiring specific training datasets.
• Investigate theoretical and empirical guarantees of accuracy preservation and pruning ratios.

Main Methods:

• Exploit adaptive data-free scalar hashing to identify redundancies in neuron weight values.
• Develop a straightforward, parallelizable algorithm to remove input-wise redundant operations in DNN layers.
• Shift computational burden from processing to efficient memory access and allocation.

Main Results:

• Demonstrate theoretical guarantees for the RED++ pruning protocol.
• Empirically validate RED++'s superiority over existing data-free pruning methods.
• Show RED++'s competitiveness with data-driven pruning techniques on various architectures (ResNets, MobileNets, EfficientNets).

Conclusions:

• RED++ offers an effective data-free approach to Deep Neural Network pruning.
• The method achieves significant inference acceleration while preserving model accuracy.
• RED++ provides a novel perspective on DNN optimization, enhancing computational efficiency.