A neural network model for timing control with reinforcement.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Humans and animals face infinite possibilities in trial-and-error learning.
• Previous work described a reward-sensitive Gaussian process (RSGP) for motor timing variability but lacked a neurobiological basis.
• A mechanistic model is needed to explain how neural circuits use reward feedback to adjust motor variability.

Purpose of the Study:

• To develop a mechanistic model of trial-and-error learning using recurrent neural networks (RNNs).
• To investigate how reward feedback influences motor variability in a Bayesian framework.
• To explore the trade-off between exploration and exploitation in continuous state control.

Main Methods:

• Utilized recurrent neural networks (RNNs) architecture to model motor timing.
• Introduced reward-dependent variability in network connectivity, inspired by synaptic transmission.
• Simulated the process within a Bayesian framework to generate output sequences.

Main Results:

• The RNN model successfully generated temporal structures of motor variability, balancing exploration and exploitation.
• The model captured both long-term correlations (memory drifts) and short-term adjustments (feedback-driven).
• The model estimated outcome uncertainty, distinguishing task-relevant variability from unexplained variability.

Conclusions:

• The proposed artificial neural network model provides a neurobiologically plausible mechanism for reward-modulated motor variability.
• This framework extends brain-inspired reinforcement learning for continuous state control.
• The model offers a better approach to understanding adjustable variability in learning and decision-making.