High-capacity neural networks on nonideal hardware.

Area of Science:

• Artificial Intelligence
• Optical Computing
• Machine Learning Hardware

Background:

• Neural networks often require high-precision hardware, limiting their application in real-world scenarios.
• Optical neural networks face challenges due to analog component inaccuracies like non-uniform illumination and nonlinear device behavior.

Purpose of the Study:

• To introduce a novel training algorithm for neural networks that compensates for hardware nonidealities.
• To demonstrate the algorithm's effectiveness on optical neural networks, specifically the Ho-Kashyap associative processor.

Main Methods:

• Developed a gated learning (off-line training) algorithm to correct for processor nonidealities.
• Utilized nonadaptive weights to reduce hardware complexity and enable high accuracy.
• Evaluated the algorithm using simulations and an optical laboratory system.

Main Results:

• Achieved a storage capacity of M/N = 1.5 on the optical system, approaching the theoretical maximum.
• Demonstrated excellent recall accuracy exceeding 95% despite hardware imperfections.
• Validated the algorithm's ability to handle nonideal analog neuron and weight accuracy.

Conclusions:

• The proposed training-out algorithm enables robust performance of neural networks on nonideal hardware.
• This approach is particularly beneficial for optical neural networks, enhancing storage capacity and accuracy.
• The techniques are adaptable to other neural network architectures and processing hardware.