Adaptive dynamic programming as a theory of sensorimotor control.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Robotics

Background:

• Optimal control theories explain sensorimotor control but often assume unrealistic knowledge of system dynamics.
• Previous models require precise system identification, which is theoretically challenging and experimentally unverified.
• A gap exists in understanding how sensorimotor systems adapt to environmental uncertainties without explicit internal models.

Purpose of the Study:

• To propose a new computational mechanism for sensorimotor control based on adaptive dynamic programming (ADP).
• To develop an iterative learning scheme within the ADP framework for motor control.
• To demonstrate the model's ability to reproduce experimental motor learning behaviors and analyze system stability.

Main Methods:

• Developed a computational model using adaptive dynamic programming (ADP), a reinforcement learning-based approach.
• Implemented an iterative learning scheme with rigorous convergence analysis.
• Validated the model by simulating motor learning in the presence of divergent and velocity-dependent force fields.

Main Results:

• The ADP-based model generates movement commands directly from real-time sensory data, bypassing the need for explicit system dynamics identification.
• The model successfully reproduced experimental findings of motor learning in altered force field environments.
• The proposed modeling strategy facilitates stability analysis of the sensorimotor control system.

Conclusions:

• Human sensorimotor systems likely employ an adaptive dynamic programming (ADP)-type mechanism for movement control.
• This ADP-based approach allows for adaptation to environmental uncertainties without requiring precise internal models of the system or environment.
• The findings provide a new perspective on how the brain achieves robust and adaptive motor control.