The gradient of the reinforcement landscape influences sensorimotor learning

Area of Science:

• Motor learning
• Reinforcement learning
• Sensorimotor adaptation

Background:

• Mastering motor skills relies on feedback from successes and failures.
• Current understanding of reinforcement learning primarily stems from discrete action experiments.
• Learning from continuous action spaces using reinforcement feedback is less understood.

Purpose of the Study:

• Investigate how humans adapt hand aiming using reinforcement feedback in continuous action spaces.
• Examine the influence of reinforcement gradient steepness on motor learning speed and strategy.
• Develop a model to explain observed motor learning behaviors.

Main Methods:

• Manipulated the reinforcement gradient (change in reward probability per change in motor action).
• Assessed participant learning speed and choice behavior under different gradient conditions.
• Compared learning on steep versus shallow gradients and opposing gradients.

Main Results:

• Participants demonstrated faster learning with steeper reinforcement gradients.
• When presented with opposing steep and shallow gradients, participants favored the steep gradient.
• A computational model was developed that aligns with the experimental findings.

Conclusions:

• The sensorimotor system utilizes gradient information for motor skill acquisition.
• Learning is enhanced by steep reinforcement gradients in continuous action spaces.
• Temporally recent and spatially local gradient information are key drivers of motor learning.