Integrating Models of Interval Timing and Reinforcement Learning

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• Reinforcement learning (RL) aims to maximize future rewards, a process critically dependent on accurate time representation in the brain.
• Distinct RL systems, including model-based and model-free, process temporal information through different neural mechanisms.

Purpose of the Study:

• To present an integrated computational view of interval timing and reinforcement learning (RL) in neural systems.
• To elucidate the distinct roles of neural systems in processing time for reward-based learning.

Main Methods:

• Computational modeling of interval timing and RL.
• Review of neurobiological evidence implicating specific brain regions and neuromodulators.

Main Results:

• Model-based RL systems utilize internal 'what happens when' models for action planning.
• Model-free RL systems employ temporal basis functions for direct reward prediction.
• A computational division of labor exists between the basal ganglia and hippocampus in processing time for RL.

Conclusions:

• The brain employs distinct neural strategies for interval timing within different reinforcement learning systems.
• Dopamine neuromodulation significantly influences how the basal ganglia and hippocampus process temporal information for reward learning.