SmartDeal: Remodeling Deep Network Weights for Efficient Inference and Training

Area of Science:

• Computer Science
• Artificial Intelligence
• Hardware Acceleration

Background:

• Deep neural networks (DNNs) demand substantial memory, leading to high energy consumption from external dynamic random-access memory (DRAM) accesses.
• Deploying DNNs on resource-constrained devices is challenging due to the energy cost of data and weight movement.

Purpose of the Study:

• To introduce SmartDeal, a hardware-friendly framework to improve DNN storage and energy efficiency for inference and training.
• To minimize energy consumption by trading memory access for computation.

Main Methods:

• Developed a novel DNN weight matrix decomposition framework with structural constraints on matrix factors.
• Decomposed weight tensors into a small basis matrix and a large, structurally sparse coefficient matrix.
• Enabled efficient storage and computation through power-of-2 quantization of non-zero elements and specialized training techniques.

Main Results:

• SmartDeal achieved up to 2.44x improvement in energy efficiency for DNN inference on real hardware.
• For DNN training, SmartDeal reduced storage by 10.56x and training energy cost by 4.48x, with negligible accuracy loss.
• Experiments covered vision and language tasks across nine models, four datasets, and three training settings.

Conclusions:

• SmartDeal offers a viable solution for energy-efficient DNN deployment and training on resource-constrained devices.
• The framework's hardware-aware design and novel decomposition technique effectively reduce memory and energy footprints.