Motion Planning-Augmented Hierarchical Reinforcement Learning for Long-Horizon Mobile Manipulation

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Long-horizon mobile manipulation involves sequential subtasks like navigation and object manipulation.
• Standard reinforcement learning faces challenges with reward sparsity and exploration.
• Hierarchical methods struggle with reliable hand-offs between subtasks due to kinematic infeasibility.

Purpose of the Study:

• To propose a motion planning-augmented hierarchical reinforcement learning architecture.
• To address sample efficiency and hand-off reliability in long-horizon mobile manipulation.
• To improve robot performance in complex indoor environments.

Main Methods:

• Decomposed tasks into subtasks using a Semi-Markov Decision Process.
• Embedded collision-free trajectories (RRT*) as reward shaping signals.
• Utilized a region-goal mechanism based on inverse kinematics for continuous hand-offs.

Main Results:

• The proposed method significantly improved subtask success rates.
• Enhanced sample efficiency compared to baseline methods.
• Performance gains compounded across the long-horizon task chain.

Conclusions:

• The motion planning-augmented hierarchical reinforcement learning architecture is effective for long-horizon mobile manipulation.
• The approach successfully resolves trade-offs between sample efficiency and hand-off reliability.
• This method offers a robust solution for complex robotic tasks in indoor settings.