The limits of counting accuracy in distributed neural representations

Learning about a causal or statistical association depends on comparing frequencies of joint occurrence with frequencies expected from separate occurrences, and to do this, events must somehow be counted. Physiological mechanisms can easily generate the necessary measures if there is a direct, one-to-one relationship between significant events and neural activity, but if the events are represented across cell populations in a distributed manner, the counting of one event will be interfered with by the occurrence of others. Although the mean interference can be allowed for, there is inevitably an increase in the variance of frequency estimates that results in the need for extra data to achieve reliable learning. This lowering of statistical efficiency (Fisher, 1925) is calculated as the ratio of the minimum to actual variance of the estimates. We define two neural models, based on presynaptic and Hebbian synaptic modification, and explore the effects of sparse coding and the relative frequencies of events on the efficiency of frequency estimates. High counting efficiency must be a desirable feature of biological representations, but the results show that the number of events that can be counted simultaneously with 50% efficiency is fewer than the number of cells or 0.1-0.25 of the number of synapses (on the two models) - many fewer than can be unambiguously represented. Direct representations would lead to greater counting efficiency, but distributed representations have the versatility of detecting and counting many unforeseen or rare events. Efficient counting of rare but important events requires that they engage more active cells than common or unimportant ones. The results suggest reasons that representations in the cerebral cortex appear to use extravagant numbers of cells and modular organization, and they emphasize the importance of neuronal trigger features and the phenomena of habituation and attention.