Apoptosis, neurogenesis, and information content in Hebbian networks

The functional significance of alternate forms of plasticity in the brain (such as apoptosis and neurogenesis) is not easily observable with biological methods. Employing Hebbian dynamics for synaptic weight development, a three-layer neural network model of the hippocampus is used to simulate nonsupervised (autonomous) learning in the context of apoptosis and neurogenesis. This learning is applied to the characters of a pair of related alphabets, first the Roman and then the Greek, resulting in a set of encodings endogenously developed by the network. The learning performance takes the form of a U-shaped curve, showing that apoptosis and neurogenesis favorably inform memory development. We also discover that networks that converge very quickly on the Roman alphabet take much longer to handle the Greek, while networks which converge over an extended timeframe can then adapt very quickly to the new language. We find that the effect becomes increasingly pronounced as the number of neurons in the dentate gyrus layer decreases, and identify a strong correlation between cases where the Roman alphabet is quickly learned and cases where a few neurons saturate many of their weights almost immediately, minimizing participation of other neurons. Cases where learning the Roman alphabet requires more time lead to larger numbers of neurons participating with a larger diversity in synaptic weights. We present an information-theoretic argument about why this implies a better, more flexible learning system and why it leads to faster subsequent correlated Greek alphabet learning, and propose that the reason that apoptosis and neurogenesis work is that they promote this effect.