A probabilistic successor representation for context-dependent learning.

Area of Science:

• Cognitive Neuroscience
• Computational Neuroscience
• Reinforcement Learning

Background:

• Learning complex tasks is hindered by uncertain and context-dependent relationships between stimuli and rewards.
• Reinforcement learning (RL) frameworks, including model-free and model-based approaches, aim to predict future rewards or states.
• The successor representation (SR) predicts future state occupancies, with theoretical links to hippocampal function in reward prediction.

Purpose of the Study:

• To develop a theory for learning SRs that accounts for uncertainty and context-dependent learning.
• To generalize SR learning to a multi-context setting, enabling the maintenance and selection of task-specific SRs.
• To unify existing SR theory with hippocampal-dependent contextual decision-making.

Main Methods:

• Introduced a theory of learning SRs using prediction errors, incorporating optimal balancing of uncertainty and existing knowledge.
• Generalized the SR learning approach to a multi-context setting, allowing inference of the appropriate SR based on prediction accuracy.
• Developed a probabilistic SR model.

Main Results:

• The proposed probabilistic SR model optimally balances uncertainty in new observations against existing knowledge.
• The model successfully learns and maintains multiple task-specific SRs in a multi-context environment.
• Contextual inference is driven by both state content and transition distributions.

Conclusions:

• The probabilistic SR model captures animal behavior in tasks requiring contextual memory and generalization.
• This work unifies previous SR theory with empirical findings on hippocampal-dependent contextual decision-making.
• The model provides a framework for understanding adaptive learning under uncertainty and changing contexts.