A sparse quantized hopfield network for online-continual memory.

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Neural networks and biological brains differ significantly in their learning mechanisms.
• Biological systems learn online from noisy, non-i.i.d. data with local synaptic plasticity.
• Deep neural networks typically use offline, i.i.d. training with non-local algorithms.

Purpose of the Study:

• To explore brain-like online learning constraints in artificial neural networks.
• To establish a standard approach for neuromorphic computing and neuroscience research.
• To propose discrete graphical models with online maximum a posteriori (MAP) learning.

Main Methods:

• Implementation of a Sparse Quantized Hopfield Network (SQHN).
• Utilizing an online maximum a posteriori (MAP) learning algorithm.
• Benchmarking against state-of-the-art neural networks on memory tasks.

Main Results:

• SQHN outperforms standard deep networks on associative memory tasks.
• SQHN shows superior performance in online, continual learning settings.
• SQHN efficiently learns from noisy inputs and excels in episodic memory tasks.

Conclusions:

• Discrete graphical models with online MAP learning offer a viable path for brain-inspired AI.
• The proposed SQHN model effectively addresses limitations of traditional deep learning.
• This approach advances understanding in both neuroscience and neuromorphic computing.