Auto-associative memory with two-stage dynamics of nonmonotonic neurons

Dynamical properties of a neural auto-associative memory with two-stage neurons are investigated theoretically. The two-stage neuron is a model whose output is determined by a two-stage nonlinear function of the internal field of the neuron (internal field is a weighted sum of outputs of the other neurons). The model is general, including nonmonotonic neurons as well as monotonic ones. Recent studies on associative memory revealed superiority of nonmonotonic neurons to monotonic ones. The present paper supplies theoretical verification on the high performance of nonmonotonic neurons and proves that the capacity of the auto-associative memory with two-stage neurons is O(n/ radicallog n), in contrast to O(n/log n) of simple threshold neurons. There is also a discussion of recall processes, where the radius of basin of attraction of memorized patterns is clarified. An intuitive explanation on why the performance is improved by nonmonotonic neurons is also provided by showing the correspondence of the recall processes of the two-stage-neuron net and orthogonal learning.