Arithmetic value representation for hierarchical behavior composition

Area of Science:

• Neuroscience
• Artificial Intelligence
• Computational Neuroscience

Background:

• Biological intelligence excels at creating new skills from existing behaviors.
• Artificial agents combine learned skills hierarchically, but brain mechanisms remain unclear.

Purpose of the Study:

• To investigate if the brain composes novel behaviors by combining pre-learned action values.
• To explore the role of behavioral stochasticity in enhancing skill composition.

Main Methods:

• Deep reinforcement learning agents were trained on composite tasks.
• Mice were pre-trained on subtasks before learning a novel composite task.
• Cortex-wide two-photon calcium imaging analyzed neural representations during learning.

Main Results:

• Deep reinforcement learning agents combined pre-learned action values additively to solve new tasks.
• Pretraining on subtasks significantly enhanced mice's learning of composite tasks.
• Neural representations mirrored combined action values, with amplified behavior variability improving learning.

Conclusions:

• The brain appears to compose novel behaviors through simple arithmetic combination of pre-acquired action-value representations.
• Stochastic policies during pretraining enhance the brain's ability to combine skills effectively.