A biologically plausible embodied model of action discovery.

Area of Science:

• Computational Neuroscience
• Reinforcement Learning
• Animal Behavior Modeling

Background:

• Animals spontaneously learn action-outcome associations to achieve goals.
• The basal ganglia, cortex, and thalamus are crucial for this learning process.
• Dopamine signaling plays a key role in reward prediction errors.

Purpose of the Study:

• To present a biologically plausible embodied model of action discovery.
• To investigate the role of dopamine and novelty salience in learning.
• To simulate and explain animal learning of action-outcome contingencies.

Main Methods:

• Developed a biomimetic model incorporating basal ganglia, cortex, and thalamus.
• Implemented reinforcement learning with dopamine signaling sensory prediction errors.
• Introduced a novelty salience mechanism linked to outcome predictability.
• Tested the model using a virtual robotic agent mimicking in vivo experiments.

Main Results:

• The model, incorporating novelty salience, successfully accounted for experimental data.
• Unpredictable outcomes led to learning when novelty salience was included.
• Predictable outcomes initially increased interactions, then decreased, facilitating learning.
• Demonstrated cortico-striatal plasticity consistent with action learning in the basal ganglia.

Conclusions:

• Action discovery relies on the interplay of neural plasticity and stimulus salience.
• The model provides a framework for understanding biological action discovery.
• Novelty salience, driven by outcome predictability, is critical for learning.