Related Experiment Videos
Reassessment of catastrophic interference.
1Department of Educational Psychology, Waseda University, Tokyo 169-8050, Japan. yamag-psy@kurenai.waseda.jp
Neuroreport
|January 11, 2005
Summary
Catastrophic interference in connectionist models is often overstated. This study found that factors like orthogonal inputs, hidden unit count, and output coding significantly impact forgetting, suggesting the problem is manageable.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Computational Neuroscience
Background:
- Connectionist models with backpropagation learning are susceptible to catastrophic interference.
- Sequential training leads to rapid performance degradation on previously learned patterns.
Purpose of the Study:
- To re-evaluate the severity of catastrophic interference in sequential training.
- To identify key factors influencing interference in connectionist models.
Main Methods:
- Conducted three sets of simulations.
- Utilized orthogonal input vectors.
- Varied the number of hidden units, output coding schemes, and input list lengths.
Main Results:
- Orthogonal input vectors resulted in mild interference.
- The number of hidden units critically affected interference levels.
- Output coding schemes and input list length were also significant factors.
Conclusions:
- The catastrophic interference problem in sequential training may be overstated.
- Careful consideration of model architecture and training parameters can mitigate interference.