The Actor-Dueling-Critic Method for Reinforcement Learning.

Area of Science:

• Robotics
• Machine Learning
• Artificial Intelligence

Background:

• Model-free reinforcement learning (RL) is crucial for robotic control.
• Traditional value function methods struggle with inaccurate Q-value estimation in stochastic environments.

Purpose of the Study:

• To enhance reinforcement learning stability and performance in noisy environments.
• To address the limitations of inaccurate Q-value estimation in stochastic settings.

Main Methods:

• Introduced the actor-dueling-critic (ADC) network, an actor-critic framework.
• Modified the critic branch to incorporate an advantage function (dueling network) for improved Q-value estimation.
• Adapted the dueling network for continuous action spaces and tested in various environments, including a custom noise environment.

Main Results:

• The ADC approach demonstrated increased stability compared to the DDPG method in noise environments.
• ADC exhibited faster convergence rates than DDPG in challenging, noisy conditions.
• The method proved effective in classic control and obstacle avoidance tasks.

Conclusions:

• The actor-dueling-critic network offers a more robust and efficient solution for reinforcement learning in robotics.
• The integration of the advantage function significantly enhances performance in environments with noise and stochasticity.
• ADC provides a promising direction for improving the reliability of autonomous systems.