Credit assignment in movement-dependent reinforcement learning.

Area of Science:

• Cognitive Neuroscience
• Decision-Making
• Reinforcement Learning

Background:

• The credit assignment problem arises when outcomes are uncertain, making it difficult to determine if failure stems from external factors or internal execution errors.
• Reinforcement learning models often simplify decision-making by assuming perfect motor control, overlooking the impact of execution variability.

Purpose of the Study:

• To investigate how motor execution failures influence decision-making and the credit assignment problem.
• To differentiate between extrinsic environmental attributions and intrinsic motor execution errors in reward-based learning.

Main Methods:

• Modified the two-armed bandit task, comparing key presses (standard) with reaching movements (allowing execution failures).
• Introduced a condition with movement error cues but outcome independence to test the role of explicit control.
• Examined individuals with cerebellar degeneration to assess implicit motor error influence.

Main Results:

• Risk aversion bias observed in key press tasks reversed in reaching tasks, suggesting motor execution impacts decisions.
• The influence of movement errors on decision-making persisted even when participants were told outcomes were independent of their movements.
• Cerebellar degeneration patients did not modulate behavior between task conditions, indicating impaired implicit error processing.

Conclusions:

• Movement execution errors implicitly influence reinforcement learning and decision-making, offering a solution to the credit assignment problem.
• A reinforcement model incorporating movement error gating provides a mechanistic explanation for observed behavioral changes.
• Findings highlight the interplay between motor control, error signals, and cognitive learning processes.