Compact and Computationally Efficient Representation of Deep Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks rely heavily on computationally intensive dot product operations.
• High computational demands limit the deployment of deep learning models on resource-constrained devices.
• Current methods like pruning and quantization yield low-entropy matrices, but existing storage formats struggle to represent them efficiently.

Purpose of the Study:

• To develop novel, efficient data structures for representing low-entropy matrices.
• To overcome the limitations of traditional sparse matrix formats in storing compressed neural network weights.
• To improve the energy and time efficiency of deep learning inference.

Main Methods:

• Proposed new matrix representations tailored for low-entropy statistics.
• Compared the proposed formats against dense and sparse representations.
• Tested performance on state-of-the-art neural networks like AlexNet, VGG-16, ResNet152, and DenseNet.

Main Results:

• Achieved up to ×42 compression ratios.
• Demonstrated up to ×5 speed ups in execution.
• Reported up to ×90 energy savings.
• The proposed formats generalize sparse formats and are more efficient.

Conclusions:

• The new data structures efficiently represent low-entropy matrices from compressed neural networks.
• Significant improvements in storage, speed, and energy efficiency were experimentally validated.
• These findings enable more efficient deployment of deep learning on edge devices.