Deep reinforcement learning trajectory planning for robotic manipulator based on simulation-efficient training.

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Robotic manipulator trajectory planning in complex environments remains challenging.
• Existing reinforcement learning methods can suffer from instability and position hopping jitter.
• Accurate inverse kinematics is crucial for precise robotic motion.

Purpose of the Study:

• To introduce a novel Multi-Actor-Critic Deep Deterministic Policy Gradient (M2ACD) algorithm for robotic manipulator trajectory planning.
• To address issues of overestimation, bias, and instability in reinforcement learning for robotics.
• To improve the smoothness and reliability of robot trajectories in complex settings.

Main Methods:

• Developed a general inverse kinematics algorithm using a Newton-MP iterative method.
• Structured the M2ACD algorithm with dual-actor and dual-critic networks to enhance stability and reduce value overestimation.
• Implemented a two-stage reward (TSR) strategy for approach and close phases of manipulation.
• Utilized Non-Uniform Rational B-Splines (NURBS) curves for trajectory smoothing.

Main Results:

• The M2ACD algorithm demonstrated superior curve smoothing, convergence stability, and convergence speed compared to TD3, DARC, and DDPG.
• Experimental validation confirmed the effectiveness of the M2ACD and the proposed kinematics algorithm.
• The M2ACD algorithm successfully mitigated position hopping jitter in trajectory planning.

Conclusions:

• The M2ACD algorithm provides an effective solution for robotic manipulator trajectory planning in complex environments.
• The proposed method enhances stability, convergence, and trajectory smoothness.
• This research lays the groundwork for advanced applications in collaborative robotics trajectory planning.