Elman backpropagation as reinforcement for simple recurrent networks.

Area of Science:

• Computational neuroscience
• Machine learning
• Artificial intelligence

Background:

• Simple recurrent networks (SRNs) are commonly trained using gradient descent-based algorithms like backpropagation (BP).
• Traditional BP requires a fully specified target answer, which is cognitively implausible for agents learning from summary feedback.
• Reinforcement learning (RL) offers a framework for learning from success/failure signals.

Purpose of the Study:

• To reimplement Elman backpropagation (BP) as a reinforcement learning (RL) scheme for SRNs.
• To demonstrate that this RL scheme yields comparable weight updates to traditional Elman BP.
• To validate the approach using network simulations on formal languages.

Main Methods:

• Developed a reinforcement learning variant of Elman backpropagation for SRNs.
• Utilized a probability interpretation of the network's output vector.
• Conducted network simulations on formal language prediction tasks.

Main Results:

• The proposed RL scheme for Elman BP shows agreement in expected weight updates with traditional Elman BP.
• Network simulations confirm the similarity in learning behaviors between Elman BP and its RL variant.
• The RL approach is effective for SRNs in symbolic time-series prediction tasks.

Conclusions:

• Elman backpropagation can be successfully reimplemented as a reinforcement learning scheme.
• This RL approach provides a more cognitively plausible learning mechanism for SRNs.
• The findings support the use of RL for training SRNs in prediction tasks with summary feedback.