Prioritized experience replay in path planning via multi-dimensional transition priority fusion

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Deep deterministic policy gradient (DDPG) algorithms for intelligent robot path planning face challenges with random experience replay, leading to inefficient training and reduced success rates.
• Current methods often overemphasize less critical experience transitions, slowing convergence and diminishing the effectiveness of path planning.

Purpose of the Study:

• To develop an improved experience transition prioritization method for DDPG-based robot path planning.
• To enhance the training process by accurately assessing the value of experience transitions.
• To accelerate algorithm convergence and increase the success rate of path planning.

Main Methods:

• Calculated experience transition priorities based on immediate reward, temporal-difference error (TD-error), and Actor network loss.
• Merged these priorities using information entropy as a weight to determine the final experience transition priority.
• Introduced adaptive adjustment for positive experience transitions and derived sampling probabilities from priorities.

Main Results:

• The proposed method demonstrated shorter test times compared to the Prioritized Experience Replay (PER) algorithm.
• Experimental results showed a reduced number of collisions with obstacles, indicating more effective path planning.
• The determined experience transition priorities accurately reflected the significance of transitions for training.

Conclusions:

• The novel prioritization method significantly enhances the utilization rate of transition replay in DDPG algorithms.
• Improved convergence speed and a higher success rate in path planning were achieved.
• This approach offers a more effective strategy for training intelligent robots in complex environments.