Related Experiment Videos
A Bayesian attractor network with incremental learning
A Sandberg1, A Lansner, K M Petersson
1Department of Numerical Analysis and Computing Science, Royal Institute of Technology, Stockholm, Sweden. asa@nada.kth.se
Network (Bristol, England)
|June 14, 2002
Summary
Online learning systems can avoid catastrophic forgetting using a palimpsest memory approach. This Bayesian confidence propagation neural network method allows gradual forgetting, improving capacity and learning speed for new data.
Area of Science:
- Artificial Intelligence
- Computational Neuroscience
- Machine Learning
Background:
- Real-time online learning systems face capacity limitations.
- Catastrophic forgetting, where old information is lost when learning new data, is a major challenge.
- Palimpsest memory, where new information overwrites old, offers a potential solution.
Purpose of the Study:
- To introduce an incremental learning rule with palimpsest properties.
- To investigate its application within an attractor neural network.
- To address the issue of catastrophic forgetting in neural networks.
Main Methods:
- Developed an incremental learning rule based on the Bayesian confidence propagation neural network.
- Implemented this rule within an attractor neural network architecture.
- Analyzed the network's learning dynamics and forgetting properties.
Main Results:
- The proposed network demonstrates palimpsest properties, avoiding catastrophic forgetting.
- Network capacity is shown to be dependent on the learning time constant.
- Faster convergence is observed for newer patterns compared to older ones.
Conclusions:
- The Bayesian confidence propagation neural network with palimpsest properties effectively mitigates catastrophic forgetting.
- This approach offers a viable solution for real-time online learning systems with finite capacity.
- The method enhances learning efficiency by prioritizing recent information.