On the Storage Capacity of Nonlinear Neural Networks

We consider the Hopfield associative memory for storing m patterns xi(r) in { - 1, + 1}(n), r = 1, em leader,m. The weights are given by the scalar product model w(ij)=(m/n)G,i not equal j,w(ii) identical with 0, where G:R --> R is some nonlinear function, like G(x) z.tbnd6; Sgn(x), which is used in hardware implementation of associative memories. We give a rigorous lower bound for the memory size of this (ANN) by using large deviations estimates. Our main results states that retrieval without errors occurs when m(n) --> infinity as n --> infinity in such a way that m(n) < (n/2log(n))q(G), where 0 < q(G): = E(NG)N))(2)/E(G(N)(2)))