Reinforcement learning via kernel temporal difference.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Reinforcement Learning

Background:

• Estimating state-action value functions is crucial for reinforcement learning.
• Adaptive filters offer a promising approach for function approximation in dynamic environments.

Purpose of the Study:

• To introduce and evaluate kernel Temporal Difference (TD)(λ), a novel kernel adaptive filter, for reinforcement learning.
• To assess the performance of kernel TD(0) in motor state decoding tasks.

Main Methods:

• Implementation of a kernel adaptive filter using stochastic gradient on temporal differences.
• Focus on the specific case of kernel TD(0).
• Comparison with a time-delay neural network (TDNN) trained via backpropagation of temporal difference error.

Main Results:

• Kernel TD(0) proved applicable for learning motor state decoding in a center-out reaching task.
• Kernel TD(0) achieved faster convergence compared to the TDNN.
• Kernel TD(0) provided a better solution than the TDNN.

Conclusions:

• Kernel TD(0) is an effective method for state-action value function estimation in reinforcement learning.
• Kernel TD(0) offers advantages in convergence speed and solution quality over TDNNs for motor control tasks.