Estimating the motor exploration in reinforcement learning.

Area of Science:

• Neuroscience
• Animal Behavior
• Computational Neuroscience

Background:

• Reinforcement learning (RL) theory models motor learning but lacks guidance on exploration strategies.
• Animal motor skill acquisition involves exploration to discover optimal behaviors.

Purpose of the Study:

• To develop a computational framework for estimating motor exploration strategies in animals.
• To investigate the role of randomness in motor learning and evolution.

Main Methods:

• Proposed a latent reinforcement learning (RL) model incorporating an explicit exploration component.
• Analyzed vocal motor variability in songbirds during pitch conditioning tasks.
• Differentiated between ideal exploration randomness and non-ideal motor variability.

Main Results:

• The latent RL model successfully decomposed songbird vocal learning into exploration and other motor components.
• Estimated vocal exploration aligned with the cortico-basal ganglia pathway's function.
• Non-ideal variability reflected suboptimal learning trajectories.

Conclusions:

• Latent RL offers a normative improvement over classical RL by making exploration explicit.
• Evolutionary pressures may favor randomness in exploration for motor learning.
• The brain's modular organization supports distinct exploration and execution mechanisms.