Rescorla-Wagner Models with Sparse Dynamic Attention.

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Machine Learning

Background:

• The Rescorla-Wagner model is a foundational theory of associative learning, explaining how agents update stimulus associations based on prediction errors.
• A key limitation of the Rescorla-Wagner model is its difficulty scaling to scenarios with a large number of predictive cues.
• Human learning suggests selective attention to cues, a mechanism not fully captured by standard Rescorla-Wagner models.

Purpose of the Study:

• To formally characterize the scaling problem of the Rescorla-Wagner model with high-dimensional cue sets.
• To propose and validate a novel framework, Sparse Attention R-W with Inference (SAR-WI), that incorporates sparse attention to address this limitation.
• To demonstrate that SAR-WI can effectively handle large cue sets while maintaining accurate associative learning.

Main Methods:

• Formal analysis of the Rescorla-Wagner model's scaling limitations.
• Development of the Sparse Attention R-W with Inference (SAR-WI) framework, integrating sparse attention mechanisms.
• Empirical evaluation of SAR-WI on natural learning tasks, comparing its performance against the standard Rescorla-Wagner model.

Main Results:

• The study formally defines the scaling problem inherent in the Rescorla-Wagner model when dealing with numerous cues.
• The proposed SAR-WI framework successfully limits attention to a sparse, relevant subset of cues.
• SAR-WI demonstrates comparable performance to the original Rescorla-Wagner model on various learning tasks, accurately inferring associative strengths and focusing on predictive cues.

Conclusions:

• Sparse attention is a viable mechanism to overcome the scaling limitations of the Rescorla-Wagner model.
• The SAR-WI framework offers a computationally tractable and effective solution for associative learning with large cue sets.
• This work provides a foundation for future research into attention mechanisms within computational models of learning.