BeMapper: BicNet and evolutionary-based multi-agent path planning with effective reinforcement

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Multi-agent path finding (MAPF) faces challenges in dynamic, crowded warehouse environments.
• Training instability and inefficient credit assignment hinder robust coordination in existing MAPF systems.

Purpose of the Study:

• To introduce BeMapper, a novel evolutionary-augmented reinforcement learning framework for improved multi-agent coordination.
• To address bottlenecks in training stability and credit assignment for MAPF in complex environments.

Main Methods:

• BeMapper integrates a multi-agent bidirectionally-coordinated network (BicNet) with a distributed actor-critic architecture.
• Key innovations include bidirectional feature fusion, evolutionary-driven critic selection, and a multi-metric scoring system for enhanced spatial awareness and accelerated convergence.
• A novel scoring system penalizes unstable behaviors and resolves credit assignment ambiguity.

Main Results:

• BeMapper achieved a 98.66% mean success rate, surpassing Mapper (95.51%) and BicNet (93.78%).
• It demonstrated a significantly higher average reward (18.81) compared to Mapper (1.65) and BicNet (0.04).
• In crowded scenarios, BeMapper reduced average travel steps to 36, outperforming competitors by 5-9 steps.

Conclusions:

• BeMapper significantly enhances coordination and robustness in multi-agent path finding for warehouse automation.
• The framework effectively improves operational throughput and stability in large-scale industrial settings.
• BeMapper represents a substantial advancement over existing state-of-the-art methods for complex MAPF problems.