Hierarchical Reinforcement Learning for Quadrupedal Robots: Efficient Object Manipulation in Constrained Environments

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Quadrupedal robots require advanced manipulation capabilities for real-world tasks.
• Operating in cluttered environments presents significant challenges for robot navigation and manipulation.
• Existing control structures often face limitations in adaptability and computational efficiency.

Purpose of the Study:

• To introduce a novel hierarchical reinforcement learning (RL) framework for object manipulation by quadrupedal robots.
• To develop a sensor-driven control structure for robust operation in dense, cluttered environments.
• To optimize decision-making using a novel, sensor-based reward function for enhanced adaptability and efficiency.

Main Methods:

• Implementation of a hierarchical reinforcement learning (RL) framework.
• Development of a sensor-driven control architecture incorporating obstacle avoidance.
• Design of a novel reward function utilizing sensor-based obstacle data.
• Simulation experiments conducted in NVIDIA Isaac Sim with ANYbotics quadrupedal robots.

Main Results:

• Achieved high object manipulation accuracy with a mean positioning error of 11 cm.
• Demonstrated successful operation across object-target distances up to 10 meters.
• Validated effective integration of path planning in complex, obstacle-filled environments.
• Showcased energy-efficient and stable robotic operations.

Conclusions:

• The proposed RL framework offers a versatile and efficient solution for quadrupedal robot object manipulation.
• The sensor-driven approach enhances robustness and adaptability in real-world, cluttered scenarios.
• This framework represents a significant advancement for practical robotic applications demanding precision and efficiency.