Partially pre-calculated weights for the backpropagation learning regime and high accuracy function mapping using...

Area of Science:

• Artificial Intelligence
• Digital Neural Networks
• Microelectronic Technology

Background:

• Artificial neural networks require accurate real-valued function mapping.
• Existing sigma-pi units have limitations in accuracy and hardware implementation.
• Gurney's sigma-pi neural model provides a foundation for structured hypercubes.

Purpose of the Study:

• To present a digital methodology for sigma-pi neural units achieving high-accuracy function mapping.
• To enable quantization of weights to 8-bits and activations to 9-bits.
• To improve accuracy by expanding the internal state space of sigma-pi units.

Main Methods:

• Partially pre-calculating weight updates in the backpropagation learning regime.
• Implementing neural units in a digital formulation with quantized weights and activations.
• Utilizing novel ring memory implementation for bit-streams instead of shift registers.
• Expanding the internal state space of sigma-pi units to increase bandwidth.

Main Results:

• Achieved accuracies better than 1% for target output functions (MSE < 0.0001).
• Demonstrated high-accuracy real-valued function mapping using RAM-based sigma-pi units.
• Successfully mapped bit-streams to RAM using ring memories for simplified hardware implementation.
• Showcased the ability of trained sigma-pi units to generalize continuous functions.

Conclusions:

• The proposed digital methodology enables highly accurate real-valued function mapping with sigma-pi neural units.
• The use of quantized weights and expanded state space offers efficient and accurate neurocomputing.
• Ring memory implementation simplifies hardware design for bit-stream processing in neural networks.