Actor-critic reinforcement learning in the songbird.

Area of Science:

• Neuroscience
• Animal Behavior
• Computational Neuroscience

Background:

• Performance learning, such as mastering a skill, shares similarities with reward learning.
• Dopamine (DA) signaling is crucial in reward-based learning and decision-making.

Purpose of the Study:

• To investigate common neural mechanisms underlying reward and performance learning.
• To explore the role of dopamine neurons in vocal learning and skill acquisition in songbirds.

Main Methods:

• Electrophysiological recordings of dopamine neuron activity in songbirds.
• Analysis of vocal variations and their reinforcement during learning.
• Modeling of reinforcement learning processes, including credit assignment and catastrophic forgetting.

Main Results:

• Dopamine neuron activation in songbirds during successful vocalizations parallels reward responses in mammals.
• DA signals reinforce vocal variations, analogous to strengthening stimulus-response associations.
• Limbic inputs to DA neurons predict song syllable quality, similar to predicting reward value.

Conclusions:

• Shared neural mechanisms, particularly involving dopamine, govern both reward and performance learning.
• Songbirds utilize reinforcement learning strategies, including node perturbation and consolidation, for vocal skill acquisition.
• An 'actor-critic' circuit motif optimizes performance quality by learning from prediction errors, mirroring reward maximization.