Approximate Policy-Based Accelerated Deep Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Reinforcement Learning

Background:

• Deep Reinforcement Learning (DRL) algorithms show high performance but suffer from slow training due to complex networks and numerous parameters, limiting learning efficiency.
• The time-consuming training process hinders the practical application of DRL agents.

Purpose of the Study:

• To accelerate the learning process of DRL agents.
• To improve the learning efficiency of deep reinforcement learning algorithms.

Main Methods:

• Propose a novel Approximate Policy-based Accelerated (APA) algorithm based on error analysis of approximate policy iteration reinforcement learning.
• Develop three new DRL algorithms: APA-DQN, APA-Double DQN, and APA-DDPG by integrating the APA algorithm with existing DRL frameworks.
• Validate the algorithms on both discrete-action and continuous-action tasks.

Main Results:

• The APA algorithm is proven to be convergent, even with higher learning rates, leading to faster DRL agent learning.
• The proposed APA-DQN, APA-Double DQN, and APA-DDPG algorithms demonstrate adaptability and superior performance compared to baseline methods.
• The accelerated algorithms show significant potential for practical applications in diverse tasks.

Conclusions:

• The APA algorithm effectively speeds up DRL training and improves learning efficiency.
• The integration of APA with DQN, Double DQN, and DDPG enhances their performance and adaptability.
• The proposed methods hold great promise for advancing the practical utility of DRL.