Progressively Learning to Reach Remote Goals by Continuously Updating Boundary Goals

Area of Science:

• Robotics
• Reinforcement Learning
• Artificial Intelligence

Background:

• Training effective policies for complex goal-reaching tasks with sparse rewards remains a significant challenge.
• Reaching remote goals (RRG) is particularly difficult due to unavailable rewards and large Wasserstein distances between goal and initial state distributions, rendering existing methods ineffective.

Purpose of the Study:

• To propose a novel method, Progressively Learning to Reach Remote Goals (PLUB), to address the challenges of RRG tasks.
• To reduce the Wasserstein distance between boundary goal and desired goal distributions for efficient policy training.

Main Methods:

• Introduced the concept of a 'boundary goal' as the set of closest achieved goals for each desired goal.
• Utilized 'closest moving distance,' an upper bound of Wasserstein distance, to reduce computational complexity.
• Developed a strategy for selecting intermediate goals and continuously updating boundary goals to minimize distances.

Main Results:

• PLUB effectively reduces both closest moving distance and Wasserstein distance.
• RRG tasks are transformed into common goal-reaching tasks solvable by hindsight relabeling and learning from demonstrations (LfD).
• Demonstrated substantial improvements over existing methods in extensive robotic manipulation experiments.

Conclusions:

• PLUB offers a robust solution for complex goal-reaching tasks with sparse rewards, particularly RRG.
• The method enhances learning efficiency by progressively reducing goal-reaching complexity.
• PLUB shows significant potential for advancing robotic manipulation and reinforcement learning applications.