Self-consistent signal-to-noise analysis of Hopfield model with unit replacement

The Hopfield model has a storage capacity: the maximum number of memory patterns that can be stably stored. The memory state of this network model disappears if the number of embedded memory patterns is larger than 0.138N, where N is the system size. Recently, it has been shown in numerical simulations that the Hopfield model with a unit replacement process, in which a small number of old units are replaced with new ones at each learning step for embedding a new pattern, can stably retrieve recently embedded memory patterns even if an infinite number of patterns have been embedded. In this paper, we analyze the Hopfield model with the replacement process by utilizing self-consistent signal-to-noise analysis. We show that 3.21 is the minimum number of replaced units at each learning step that avoids an overload evoking disappearance of the memory state when embedding an infinite number of patterns. Furthermore, we show that the optimal number of replaced units at each learning step that maximizes the number of retrievable patterns is 6.95. These critical numbers of replaced units are independent of the system size N. Finally, we compare this model with the Hopfield model with the forgetting process.