Competitive plasticity to reduce the energetic costs of learning

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Metabolic energy expenditure

Background:

• Brain function is limited by the energy required for computation and memory formation.
• Even simple learning tasks incur significant metabolic costs due to synaptic updates.
• High accuracy learning, like MNIST to 95%, demands substantial synaptic modifications.

Purpose of the Study:

• To investigate the energy demands of learning in feedforward neural networks.
• To develop energy-saving algorithms for synaptic plasticity.
• To bridge the understanding between artificial and biological learning mechanisms.

Main Methods:

• Developed a parsimonious energy model for neural network learning.
• Proposed two plasticity-restricting algorithms: 1) modifying only large synaptic updates, and 2) restricting plasticity to essential network paths.
• Compared algorithm performance against standard backpropagation.

Main Results:

• Plasticity restriction significantly reduces metabolic energy expenditure during learning.
• Algorithms achieve substantial energy savings with only a minor increase in learning time.
• Biological networks, often larger than necessary, can benefit from such plasticity constraints.

Conclusions:

• Restricting synaptic plasticity is an effective strategy for conserving metabolic energy in learning.
• These findings enhance understanding of biological plasticity and improve artificial learning models.
• The proposed algorithms offer potential benefits for energy-efficient hardware.