Cognitively inspired reinforcement learning architecture and its application to giant-swing motion control

Area of Science:

• Artificial Intelligence
• Machine Learning
• Cognitive Science

Background:

• Human cognition inspires AI algorithms, including reinforcement learning mechanisms.
• The loosely symmetric (LS) value function models human causal intuition for the exploration-exploitation dilemma.
• Existing LS methods are limited to single-state reinforcement learning problems.

Purpose of the Study:

• To propose a novel LS-Q learning architecture for general reinforcement learning tasks with multiple states and delayed rewards.
• To evaluate the performance of LS-Q learning in complex, real-world robotic applications.

Main Methods:

• Developed the LS-Q learning architecture, extending LS value functions to multi-state environments.
• Tested LS-Q learning on a giant-swing robot motion learning task without pre-existing models or state-space approximations.
• Compared LS-Q learning performance against standard Q-learning.

Main Results:

• LS-Q learning successfully acquired giant-swing motion, unlike standard Q-learning which became trapped in local optima.
• LS-Q learning demonstrated superior performance in environments with fewer, coarser states and incomplete state observations.
• The proposed method showed robust performance with minimal parameter tuning and learning time.

Conclusions:

• The LS-Q learning architecture effectively handles complex reinforcement learning tasks with multiple states and delayed rewards.
• The human-cognition-inspired LS-Q learning method shows adaptive capabilities in real-world, uncertain environments.
• LS-Q learning offers a promising approach for AI that leverages human-like causal reasoning.