High-capacity neural networks on nonideal hardware

We present a new training-out algorithm for neural networks that permits good performance on nonideal hardware with limited analog neuron and weight accuracy. Optical neural networks are emphasized with the error sources including nonuniform beam illumination and nonlinear device characteristics. We compensate for processor nonidealities during gated learning (off-line training); thus our algorithm does not require real-time neural networks with adaptive weights. This permits use of high-accuracy nonadaptive weights and reduced hardware complexity. The specific neural network we consider is the Ho-Kashyap associative processor because it provides the largest storage capacity. Simulation results and optical laboratory data are provided. The storage measure we use is the ratio M/N of the number of vectors stored (M) to the dimensionality of the vectors stored (N). We show a storage capacity of M/N = 1.5 on our optical laboratory system with excellent recall accuracy, > 95%. The theoretical maximum storage is M/N = 2 (as N approaches infinity), and thus the storage and performance we demonstrate are impressive considering the processor nonidealities we present. Our techniques can be applied to other neural network algorithms and other nonideal processing hardware.