RCT: Resource Constrained Training for Edge AI.

Area of Science:

• Artificial Intelligence
• Computer Science
• Sustainable Computing

Background:

• Efficient neural network training is crucial for edge AI and reducing computational carbon footprints.
• Current training methods demand substantial memory and energy, exceeding edge device capabilities.
• Large data movement between off-chip and on-chip memory leads to significant energy waste.

Purpose of the Study:

• To introduce Resource Constrained Training (RCT) for efficient neural network training on resource-limited edge devices and servers.
• To reduce memory footprint and energy consumption during the training process.
• To enable effective in situ training of AI models on edge hardware.

Main Methods:

• RCT maintains a quantized model throughout training, minimizing memory requirements for parameters.
• Per-layer bitwidth is dynamically adjusted to conserve energy when lower precision suffices.
• Experiments were conducted on image classification, natural language processing, and crowd counting tasks.

Main Results:

• Average 8-15 bit weight updates achieve state-of-the-art (SOTA) performance across diverse applications.
• RCT reduces memory for model parameters by 63.5%-80% and cuts communication energy.
• Contrary to common compression practices, efficient training does not benefit from special treatment of first/last layers.

Conclusions:

• RCT offers a viable solution for resource-efficient neural network training on edge devices.
• Dynamic bitwidth adjustment and quantization are key to reducing memory and energy demands.
• Dataset complexity influences optimal bitwidth, with harder datasets requiring lower precision for efficient training.