Human reinforcement learning subdivides structured action spaces by learning effector-specific values.

Area of Science:

• Neuroscience
• Cognitive Science
• Reinforcement Learning

Background:

• High-dimensional action spaces pose a challenge for reinforcement learning.
• Neural systems must efficiently learn action values despite numerous effectors.

Purpose of the Study:

• To investigate if the brain decomposes action values into effector-specific components.
• To test a reinforcement learning framework for handling high-dimensional action spaces.

Main Methods:

• Subjects performed a multieffector choice task with simultaneous left and right hand decisions.
• Behavioral data and blood oxygen-level-dependent (BOLD) activity were analyzed.
• Computational models compared decomposed versus unitary action value representations.

Main Results:

• Human behavior favored a learning model with decomposed action values.
• BOLD signaling showed effector-specific value decomposition in the striatum and intraparietal sulcus.
• Lateralized biases reflected prediction errors and value correlates.

Conclusions:

• The brain employs decomposed value representations to manage reinforcement learning in high-dimensional action spaces.
• This 'divide and conquer' strategy enhances behavioral flexibility and learning efficiency.