Hierarchical Deep Reinforcement Learning for Continuous Action Control.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Robotic control in continuous action spaces presents significant challenges, particularly for complex tasks requiring both basic and compound skill acquisition.
• Existing methods often struggle to efficiently learn these hierarchical skill sets.

Purpose of the Study:

• To propose a novel hierarchical deep reinforcement learning algorithm for simultaneous learning of basic and compound robotic skills.
• To address the limitations of current approaches in continuous action spaces for complex robotic tasks.

Main Methods:

• A two-level hierarchical reinforcement learning architecture was developed.
• The first level utilizes individual actors for basic skills supervised by a shared critic.
• The second level employs a meta-critic to learn compound skills by reusing basic skills.

Main Results:

• The algorithm successfully learned high-performance basic and compound skills concurrently on a Pioneer 3AT robot.
• Evaluated in Gazebo 2 simulations across three navigation scenarios.
• Learned compound skills demonstrated superior performance compared to a discrete action space deep reinforcement learning algorithm.

Conclusions:

• The proposed hierarchical deep reinforcement learning approach effectively enables robots to learn both fundamental and complex skills simultaneously.
• This method offers a promising solution for enhancing robotic capabilities in continuous action spaces.
• The algorithm's performance indicates a significant advancement over discrete action space methods for robotic task learning.