Heuristic dense reward shaping for learning-based map-free navigation of industrial automatic mobile robots

This paper presents a map-free navigation approach for industrial automatic mobile robots (AMRs), designed to ensure computational efficiency, cost-effectiveness, and adaptability. Utilizing deep reinforcement learning (DRL), the system enables real-time decision-making without fixed markers or frequent map updates. The central contribution is the Heuristic Dense Reward Shaping (HDRS), inspired by potential field methods, which integrates domain knowledge to improve learning efficiency and minimize suboptimal actions. To address the simulation-to-reality gap, data augmentation with controlled sensor noise is applied during training, ensuring robustness and generalization for real-world deployment without fine-tuning. Training results underscore HDRS's superior convergence speed, training stability, and policy learning efficiency compared to baselines. Simulation and real-world evaluations establish HDRS-DRL as a competitive alternative, outperforming traditional approaches, and offering practical applicability in industrial settings.