Integrating temporal difference methods and self-organizing neural networks for reinforcement learning with delayed...

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Robotics

Background:

• Autonomous agents require robust learning mechanisms for dynamic environments.
• Integrating multimodal sensory inputs, actions, and rewards is crucial for effective decision-making.
• Existing reinforcement learning systems often struggle with delayed feedback and computational efficiency.

Purpose of the Study:

• To present a novel neural architecture, TD-FALCON, for learning category node encodings across multimodal patterns.
• To enable autonomous agents to adapt and function effectively using immediate and delayed reinforcement signals.
• To enhance learning speed, task completion, and efficiency in agent navigation.

Main Methods:

• Developed the TD fusion architecture for learning, cognition, and navigation (TD-FALCON).
• Integrated adaptive resonance theory (ART) and temporal difference (TD) learning methods.
• Utilized on-policy (SARSA) and off-policy (Q-learning) TD methods to estimate value functions.
• Implemented an action selection policy based on learned value functions.

Main Results:

• TD-FALCON systems demonstrated effective learning with both immediate and delayed reinforcement.
• Achieved stable performance significantly faster than standard gradient-descent-based reinforcement learning systems.
• Exhibited superior task completion rates and efficiency in a minefield navigation task.

Conclusions:

• TD-FALCON provides an efficient neural architecture for reinforcement learning in dynamic environments.
• The integration of ART and TD learning enhances an agent's adaptability and learning speed.
• TD-FALCON shows promise for developing more capable and efficient autonomous agents.