An Improved Prioritized DDPG Based on Fractional-Order Learning Scheme.

Area of Science:

• Reinforcement Learning
• Machine Learning
• Robotics

Background:

• Deep deterministic policy gradient (DDPG) is effective for large-scale continuous control.
• However, DDPG suffers from low sample utilization efficiency and insufficient exploration.

Purpose of the Study:

• To address the limitations of the standard DDPG algorithm.
• To improve training convergence speed, sample efficiency, and exploration capabilities.

Main Methods:

• Introduced a fractional gradient optimizer to enhance training speed and accuracy.
• Implemented high-value experience replay with weight-changed priority for improved sample efficiency.
• Adopted an optimized exploration strategy for boundary action spaces to enhance environmental exploration.

Main Results:

• The proposed method significantly speeds up the learning process.
• Achieved higher average rewards compared to existing DDPG algorithms.
• Demonstrated effectiveness on the gym and pybullet platforms.

Conclusions:

• The enhanced DDPG algorithm effectively overcomes the limitations of the standard DDPG.
• The improvements lead to faster convergence and superior performance in continuous control tasks.
• The proposed method offers a promising advancement for reinforcement learning applications.