Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation.

Area of Science:

• Robotics and Artificial Intelligence
• Machine Learning for Autonomous Systems

Background:

• Reinforcement learning (RL) shows promise for unmanned ground vehicles (UGVs) but faces challenges due to limited computing resources and difficulties in training navigation tasks.
• Current RL methods exhibit limited intelligence, often failing in corner cases and requiring extensive, carefully designed reward functions and interactions.
• The gap between simulation training and real-world deployment (sim-to-real gap) hinders the practical application of RL in UGVs.

Purpose of the Study:

• To propose a human-guided reinforcement learning (RL) framework to enhance the performance of unmanned ground vehicles (UGVs) in navigation.
• To improve RL capabilities during both simulation learning and real-world deployment by integrating human intelligence and intervention.
• To address the limitations of current RL methods in terms of computational resources, training complexity, and robustness in diverse environments.

Main Methods:

• Developed a novel human-guided RL algorithm incorporating a human-guided learning objective, prioritized human experience replay, and human intervention-based reward shaping.
• Employed a denoised representation for domain adaptation to mitigate the simulation-to-real gap, enabling effective transfer learning from simulation to the real world.
• Validated the approach through extensive simulations and real-world experiments using UGVs with minimal neural networks and image-based inputs.

Main Results:

• The human-guided RL framework significantly improved UGV navigation performance in diverse and dynamic environments, outperforming existing learning- and model-based approaches.
• The method demonstrated superior goal-reaching capabilities and enhanced safety compared to traditional RL navigation techniques.
• The approach proved robust to variations in input features and ego-kinetics, and enabled online learning of desired behaviors using small-scale human demonstrations.

Conclusions:

• Combining human intelligence with reinforcement learning offers a powerful solution for developing capable and robust navigation systems for unmanned ground vehicles.
• The proposed human-guided RL framework effectively overcomes the limitations of conventional RL, enabling efficient learning and deployment in real-world scenarios.
• This approach paves the way for more intelligent and adaptable autonomous navigation systems, even with constrained computational resources.