Hybrid Precision Floating-Point (HPFP) Selection to Optimize Hardware-Constrained Accelerator for CNN Training.

Area of Science:

• Artificial Intelligence
• Computer Architecture
• Hardware Acceleration

Background:

• Edge devices require efficient AI accelerators due to high hardware costs of floating-point operations.
• Traditional block floating-point (BFP) formats like MSFP and FlexBlock have limited dynamic range and precision, hindering deep neural network (DNN) training.
• Existing reduced precision formats often use FP32 for accumulation, limiting hardware savings.

Purpose of the Study:

• To introduce hybrid precision (HPFP) selection algorithms to address limitations of traditional floating-point formats in AI accelerators.
• To balance layer-wise arithmetic operations and data path precision for efficient DNN training on edge devices.
• To achieve significant hardware savings by performing all multiply and accumulate operations in reduced floating-point format.

Main Methods:

• Developed HPFP selection algorithms for systematic precision reduction and hybrid precision strategies.
• Implemented two training accelerators for YOLOv2-Tiny using distinct mixed precision strategies.
• Benchmarked HPFP against a conventional Bfloat16 accelerator.

Main Results:

• HPFP with 10-bit data path and 12-bit for higher precision layers achieved a 49.4% reduction in energy consumption.
• Memory access decreased by 37.5% with the HPFP approach.
• A marginal mean Average Precision (mAP) degradation of 0.8% was observed compared to Bfloat16.

Conclusions:

• The proposed HPFP selection algorithms enable efficient design of compact, low-power AI accelerators for edge devices.
• HPFP offers a viable solution for reducing hardware costs and energy consumption without significant accuracy compromise.
• This approach facilitates effective DNN training on resource-constrained edge devices.