The relationship of local context codes to sequence length memory capacity

This paper pursues part of a theory to understand sequence prediction networks that use local context neuron encodings. In particular, a previously described neural network model of hippocampal CA3 is studied. General expressions relating CA3 interpattern distances (which reflect local context codes) to sequence length memory capacity are created and verified by computer simulations. As a result, we confirm a very simple relationship between the sequence length memory capacity and a combination of average activity level and the average local context lifetimes. Sequence length memory capacity is also bounded for networks using such local context neuronal codes. Thus, this simple theory quantifies an important limitation on a fully adaptive (i.e. self-supervising) neural model capable of creating context-dependent learning and memory.