Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms.

Area of Science:

• Reinforcement Learning
• Machine Learning
• Robotics

Background:

• Traditional Deep Deterministic Policy Gradient (DDPG) algorithms struggle with local optima and significant error fluctuations in continuous action spaces.
• These limitations hinder the effectiveness of DDPG in complex control tasks.

Purpose of the Study:

• To propose a novel Dual-Actor-Dual-Critic DDPG (DN-DDPG) algorithm to overcome the limitations of the traditional DDPG.
• To enhance the stability and performance of reinforcement learning agents in continuous action spaces.

Main Methods:

• Implemented a dual-critic network architecture, selecting the minimum Q-value to mitigate local optima.
• Introduced a dual-actor network, choosing the action with the maximum value to stabilize training and address value underestimation.
• Validated the DN-DDPG algorithm on four continuous control tasks using the MuJoCo simulation environment.

Main Results:

• The DN-DDPG algorithm demonstrated a reduced range of error fluctuations compared to the standard DDPG.
• The proposed method achieved a higher cumulative return, indicating improved learning performance.
• The dual-actor-dual-critic approach effectively stabilized the training process.

Conclusions:

• The DN-DDPG algorithm offers a robust solution for improving DDPG performance in continuous action spaces.
• This enhancement is crucial for applications requiring stable and efficient reinforcement learning agents, such as robotics and autonomous systems.