Inferring learning rules during de novo task learning

Area of Science:

• Neuroscience
• Computational Neuroscience
• Animal Behavior

Background:

• Identifying learning rules governing behavior is a key neuroscience challenge.
• Reinforcement learning (RL) provides a framework, but studies often use non-stationary environments, not de novo learning.
• Understanding how animals acquire entirely new tasks is crucial.

Purpose of the Study:

• To introduce a statistical framework for inferring RL rules directly from single-animal behavior.
• To compare policy-gradient-like rules with classical temporal-difference algorithms for de novo task learning.
• To uncover systematic deviations from standard RL models in animal learning.

Main Methods:

• Developed a statistical framework to infer RL rules from behavioral data.
• Applied the framework to mice learning a perceptual decision-making task.
• Fitted flexible parametric learning rules to behavioral data.

Main Results:

• Policy-gradient-like rules better explain de novo task learning than temporal-difference algorithms.
• Identified deviations from standard RL, including side-specific learning rates and negative reward baselines.
• Discovered that animals adapt learning rates dynamically over training and across curricula.

Conclusions:

• The framework provides a statistical account of how animals learn new tasks from scratch.
• Animal learning exhibits key departures from classical reinforcement learning algorithms.
• Findings offer insights into the neural mechanisms underlying adaptive learning.