An Improved Dyna-Q Algorithm Inspired by the Forward Prediction Mechanism in the Rat Brain for Mobile Robot Path...

Area of Science:

• Robotics
• Neuroscience
• Artificial Intelligence

Background:

• Traditional Model-Based Reinforcement Learning (MBRL) faces challenges in computational cost, convergence, and performance for robot spatial cognition and navigation.
• Existing MBRL algorithms struggle to explain rapid adaptation and complex task learning observed in animal behavior.
• Vicarious trial and error (VTE) and hippocampal forward prediction mechanisms in mammals offer insights for goal-oriented behavior.

Purpose of the Study:

• To develop an improved Dyna-Q algorithm addressing limitations of traditional MBRL.
• To incorporate the hippocampus forward prediction mechanism into MBRL for enhanced robot navigation.
• To tackle the exploration-exploitation dilemma in Reinforcement Learning (RL).

Main Methods:

• Proposed an improved Dyna-Q algorithm inspired by the hippocampus forward prediction mechanism.
• Algorithm alternates potential future paths and dynamically adjusts sweep length based on decision certainty for action selection.
• Tested the algorithm in a 2D maze with static and dynamic obstacles.

Main Results:

• The enhanced algorithm demonstrated faster spatial cognition compared to classic RL algorithms like SARSA and Dyna-Q.
• Improved global search ability in path planning was observed.
• The method effectively addressed the exploration-exploitation dilemma in RL tasks.

Conclusions:

• The proposed algorithm offers a biologically inspired approach to enhance MBRL for robot navigation.
• It provides a new perspective for spatial cognitive tasks by integrating brain mechanisms.
• The method effectively solves complex navigation tasks, reflecting how the brain organizes MBRL.