Optimised weight programming for analogue memory-based deep neural networks

Area of Science:

• Artificial Intelligence
• Computer Engineering
• Materials Science

Background:

• Analogue memory deep neural networks offer significant energy and throughput advantages over digital systems like GPUs.
• Current research emphasizes hardware-aware training and component improvements.
• Translating software weights to analogue hardware, accounting for memory imperfections, is a critical challenge.

Purpose of the Study:

• To develop a generalized computational framework for automating weight programming strategies in analogue memory deep neural networks.
• To minimize accuracy degradation during inference, especially over time.
• To enable analogue accelerators to achieve their full inference potential.

Main Methods:

• A generalized computational framework was developed to automate complex weight programming strategies.
• The approach uses a flexible numerical heuristic to accommodate device-level complexities.
• The framework was tested for its generalizability across different neural network architectures.

Main Results:

• The framework successfully minimizes accuracy degradations during inference in analogue memory deep neural networks.
• It demonstrates generalizability across recurrent, convolutional, and transformer neural network structures.
• The approach accommodates arbitrary device-level complexities in analogue memories.

Conclusions:

• The developed framework automates weight programming for analogue memory deep neural networks, improving inference accuracy and reliability.
• This method is adaptable to various analogue memory types and neural network architectures.
• It provides a pathway to unlock the full performance potential of analogue deep neural network accelerators.