Reinforcement Learning-Based Reactive Obstacle Avoidance Method for Redundant Manipulators.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Redundant manipulators are crucial for human-robot collaboration, offering flexibility.
• Ensuring obstacle avoidance while maintaining trajectory tracking is vital for manipulator efficiency and safety.
• Traditional methods often face challenges like joint singularity and exceeding position limits.

Purpose of the Study:

• To propose a reactive obstacle avoidance method for redundant manipulators by integrating deep reinforcement learning (DRL) with the gradient projection method.
• To develop a DRL framework enabling a reinforcement learning agent to learn motion within the null space of the manipulator's Jacobian matrix.
• To enhance the reward function for automatic constraint handling, including maintaining manipulability and avoiding joint singularity.

Main Methods:

• Integration of deep reinforcement learning (DRL) with the gradient projection method.
• Development of a general DRL framework for obstacle avoidance tasks.
• Application of a reinforcement learning agent to learn null-space motion.
• Redesigning the reward function to incorporate the manipulability index for constraint satisfaction.

Main Results:

• The proposed DRL-integrated method demonstrates superior performance compared to the conventional gradient projection method.
• Achieved higher success rates in obstacle avoidance.
• Maintained better average manipulability, effectively avoiding joint singularities.
• Showed improved time efficiency in simulations with a 4-degree-of-freedom planar manipulator.

Conclusions:

• The DRL-enhanced gradient projection method offers an effective solution for obstacle avoidance in redundant manipulators.
• The approach successfully addresses limitations of conventional methods, improving safety and performance.
• The integration of manipulability into the reward function is key to maintaining dexterity and avoiding singularities.