Learning With Stochastic Guidance for Robot Navigation.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Reinforcement learning (RL) methods like deep deterministic policy gradient (DDPG) struggle with high variance in complex environments due to sparse rewards and environmental variations.
• Existing DDPG approaches often require extensive training and can be inefficient in real-world applications.

Purpose of the Study:

• To present a new framework that mitigates high variance issues in DDPG by incorporating a stochastic switch.
• To enable agents to dynamically select between high- and low-variance policies for improved learning efficiency.
• To bootstrap robot navigation capabilities quickly by leveraging heuristic controllers.

Main Methods:

• A novel framework is proposed featuring a stochastic switch that can be jointly trained with the deep deterministic policy gradient (DDPG) algorithm.
• The framework allows a reinforcement learning agent to dynamically choose between exploration (high-variance policy) and heuristic guidance (low-variance policy).
• The approach was demonstrated on a robot navigation task, integrating multiple simple independent controllers for initial guidance.

Main Results:

• The proposed framework effectively and efficiently trains DDPG navigation policies.
• Robots utilizing the stochastic guidance achieved significantly better performance compared to state-of-the-art baseline models.
• The navigation capability of robots was rapidly bootstrapped, avoiding completely random initial actions.

Conclusions:

• The stochastic switch framework successfully addresses the high variance problem in DDPG for complex tasks.
• This approach offers a more efficient and effective method for training reinforcement learning agents in real-world scenarios.
• The integration of heuristic guidance significantly improves robot navigation performance and learning speed.