Using Reinforcement Learning to Investigate Neural Dynamics During Motor Learning.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Motor Control

Background:

• Motor cortex preparatory activity shifts during motor learning, hypothesized to underlie memory.
• Understanding these shifts is key to deciphering motor memory acquisition, retention, and retrieval.

Purpose of the Study:

• To train recurrent neural network (RNN) models simulating motor learning phenomena.
• To investigate how reinforcement learning (RL) and prior experience shape neural activity during motor adaptation.

Main Methods:

• Developed a curl field (CF) motor learning task environment.
• Trained RNNs using RL with novel regularization for realistic reaching trajectories.
• Analyzed model preparatory activity for stability, subspace, and orthogonality across learning, washout, and relearning.

Main Results:

• Trained RNNs reproduced monkey behavioral findings in CF adaptation without supervision.
• Model preparatory activity resided in a stable, force-predictive subspace.
• Washout shifts became more orthogonal to learning shifts with pretraining, mimicking brain activity.

Conclusions:

• RL-trained RNNs can model motor learning and neurophysiological findings.
• Prior experience may organize preparatory neural activity by forming structured dynamical motifs, enhancing brain-like characteristics.