Value-Based Subgoal Discovery and Path Planning for Reaching Long-Horizon Goals

Area of Science:

• Robotics
• Artificial Intelligence
• Reinforcement Learning

Background:

• Autonomous agents face challenges in achieving long-horizon goals in spatial traversal.
• Current subgoal graph methods use arbitrary heuristics and struggle with erroneous subgoal connections, especially across obstacles.

Purpose of the Study:

• To propose a novel subgoal graph-based planning method, LSGVP, for improved long-horizon goal achievement in autonomous agents.
• To address limitations of existing methods regarding subgoal sampling, discovery, and graph pruning.

Main Methods:

• Introduced LSGVP, a method utilizing value-based subgoal discovery for sparse subgoal generation aligned with cumulative reward.
• Implemented automatic pruning of the learned subgoal graph to eliminate erroneous connections.

Main Results:

• LSGVP yields subgoals on higher cumulative reward paths.
• The method demonstrates higher cumulative rewards compared to other heuristics.
• LSGVP achieves superior goal-reaching success rates over state-of-the-art methods.

Conclusions:

• LSGVP effectively addresses challenges in long-horizon planning for autonomous agents.
• The combination of value-based discovery and automatic pruning enhances performance in spatial traversal tasks.