Probabilistic metaplasticity for continual learning with memristors in spiking networks

Area of Science:

• Artificial Intelligence
• Neuroscience
• Materials Science

Background:

• Edge devices require continual learning capabilities for dynamic environments.
• Resource constraints on edge devices challenge traditional continual learning methods due to memory and computational overhead.
• Memristor-based crossbar architectures offer energy-efficient compute-in-memory but suffer from low precision, hindering their use in precise weight modulation for continual learning.

Purpose of the Study:

• To propose a novel continual learning mechanism for resource-constrained edge devices.
• To overcome the limitations of low-precision memristor devices in continual learning.
• To reduce memory overhead and energy dissipation associated with continual learning on edge devices.

Main Methods:

• Introduced probabilistic metaplasticity, a mechanism that modulates weight update probability instead of magnitude.
• Integrated probabilistic metaplasticity into a spiking neural network utilizing low-precision memristor weights.
• Evaluated the model on continual learning benchmarks.

Main Results:

• Probabilistic metaplasticity achieved performance comparable to state-of-the-art continual learning models using high-precision weights.
• The proposed mechanism eliminated the need for auxiliary high-precision memory.
• Demonstrated ~67% lower memory for additional parameters and up to ~60x lower energy during parameter updates compared to auxiliary memory solutions.

Conclusions:

• Probabilistic metaplasticity offers an effective solution for energy-efficient continual learning on edge devices with low-precision emerging hardware.
• The method significantly reduces memory and energy overhead, making advanced AI capabilities more feasible on resource-constrained platforms.
• This work paves the way for practical, low-power continual learning systems leveraging memristor technology.