Reduction of required precision bits for back-propagation applied to pattern recognition.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Engineering

Background:

• Hardware implementations of backpropagation (BP) face limitations in precision bits for operations and data.
• Reducing rounding errors is critical for effective BP in resource-constrained hardware.
• Conventional BP minimizes squared error, which can lead to underflow and weight update omissions.

Purpose of the Study:

• To develop a new learning algorithm for backpropagation that mitigates underflow and omission errors in low-precision hardware.
• To enhance the accuracy and efficiency of neural network training with limited precision bits.
• To reduce the memory requirements for weight storage in BP implementations.

Main Methods:

• Introduced a novel learning algorithm based on the overestimation of significant error.
• The new algorithm minimizes a weighted error function, differing from the conventional squared error minimization.
• Simulations were conducted using multifont capital recognition tasks.

Main Results:

• The new algorithm achieved 100% recognition accuracy with 8-bit precision.
• It demonstrated high recognition accuracy (94.9%) for unseen characters.
• Performance with 8-bit precision was comparable to conventional BP using 12-bit precision.

Conclusions:

• The weighted error function effectively alleviates underflow and omission issues in low-precision BP.
• The algorithm shows robust performance even with a reduced number of hidden neurons.
• This approach significantly reduces the required amount of weight memory for BP implementations.