Sparse coding with a somato-dendritic rule | JoVE Visualize

Area of Science:

Computational neuroscience
Artificial intelligence
Machine learning

Background:

Cortical neurons exhibit sparse activity, crucial for low-energy brain computation and efficient information processing.
Sparse coding offers advantages for associative learning, balancing information storage capacity and readout ease.
Existing methods for learning sparse codes, like auto-encoders and single-layer networks, often rely on homeostatic plasticity, risking catastrophic forgetting in continuous learning systems.

Purpose of the Study:

To investigate if plasticity at recurrent inhibitory synapses can regulate neural population sparseness and individual neuron firing rates.
To explore an alternative to homeostatic plasticity for achieving sparse coding in artificial neural networks.
To demonstrate the efficacy of a somato-dendritic learning rule in a compartmentalized neural network model.

Main Methods:

Developed a neural network model with compartmentalized inputs: rate-based dendritic compartments for feedforward integration and spiking integrate-and-fire somas for recurrent inhibition.
Implemented a somato-dendritic learning rule enabling somatic inhibition to modulate nonlinear Hebbian learning in dendrites.
Trained the network on MNIST digits and natural images to learn sparse representations.

Main Results:

The network successfully discovered independent components, forming a sparse encoding of the input data.
The learned sparse encoding supported linear decoding, demonstrating effective information representation.
Inhibitory synaptic plasticity was shown to effectively regulate population sparseness and neuron firing rates, similar to homeostatic plasticity.

Conclusions:

Intrinsic homeostatic plasticity is not the sole mechanism for regulating neural sparseness; inhibitory synaptic plasticity can achieve the same outcome.
Compartmentalized neural network designs offer advantages for implementing complex learning rules and achieving efficient computation.
The study advocates for moving beyond simple point neuron models towards more biologically plausible compartmentalized models in artificial spiking neural networks.