LLMs augmented hierarchical reinforcement learning with action primitives for long-horizon manipulation tasks.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Deep reinforcement learning (RL) faces challenges with long-horizon manipulation tasks due to large state spaces and sparse rewards.
• Hierarchical RL improves skill learning but struggles with training efficiency and transferability.
• Large language models (LLMs) offer world knowledge and reasoning but lack real-world task grounding.

Purpose of the Study:

• To develop a novel approach that combines the planning capabilities of LLMs with RL for long-horizon manipulation tasks.
• To improve sample efficiency and performance in complex robotic tasks by integrating LLMs into a hierarchical RL framework.

Main Methods:

• Proposed a hierarchical agent, LARAP (LLM-guided hierarchical agent with parameterized action primitives).
• Utilized LLMs to guide a high-level policy, enhancing sample efficiency during training.
• Combined LLMs with parameterized action primitives for long-horizon manipulation.

Main Results:

• LARAP significantly outperformed baseline methods in various simulated manipulation tasks.
• The approach demonstrated improved sample efficiency compared to traditional RL methods.
• The integration of LLMs effectively guided the hierarchical agent's learning process.

Conclusions:

• LARAP offers a promising solution for addressing long-horizon manipulation challenges in robotics.
• Combining LLMs with RL provides a powerful framework for complex task learning.
• The LARAP agent shows potential for real-world applications requiring sophisticated manipulation skills.