Goal representation heuristic dynamic programming on maze navigation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Markov decision processes (MDPs) are fundamental to reinforcement learning.
• Traditional methods often rely solely on external reinforcement signals.
• Heuristic Dynamic Programming (HDP) uses an actor-critic design for value function approximation.

Purpose of the Study:

• To introduce Goal Representation Heuristic Dynamic Programming (GrHDP) for improved online learning in MDPs.
• To develop an adaptive internal goal/reward representation for agents.
• To enhance value function approximation through a novel goal network.

Main Methods:

• The proposed GrHDP algorithm maintains the actor-critic structure of HDP.
• A goal network is integrated to generate an internal goal signal.
• The algorithm is evaluated on 2-D and 3-D maze navigation tasks.

Main Results:

• GrHDP demonstrates improved learning performance compared to traditional HDP.
• Performance is benchmarked against Sarsa(λ) and Q-learning algorithms.
• Convergence analysis of neural network weights provides theoretical guarantees.

Conclusions:

• GrHDP offers a significant advancement in online learning for MDPs.
• The internal goal representation effectively aids value function approximation.
• The method shows promise for complex navigation tasks and provides theoretical validation.