Dynamic-layer transformer-based reinforcement learning for observation-constrained multi-agent roundup scenarios.

Area of Science:

• Robotics
• Artificial Intelligence
• Multi-Agent Systems

Background:

• Cooperative multi-agent systems face challenges in coordination under limited perception.
• Decentralized partially observable Markov decision processes (Dec-POMDPs) are complex for real-world applications.

Purpose of the Study:

• To propose TransMARL, a novel transformer-based multi-agent reinforcement learning framework.
• To enable effective coordination in observation-constrained multi-agent roundup tasks.

Main Methods:

• Formulated the roundup task as a Dec-POMDP with local observations and dynamic interaction graphs.
• Developed a framework combining graph feature encoding and policy execution for decentralized decision-making.
• Designed a task-informed reward function promoting coverage, approach, uniformity, and collision avoidance.
• Adaptively adjusted transformer depth based on team size to balance performance and computational cost.

Main Results:

• TransMARL demonstrated competitive and improved performance against baselines in simulation.
• The framework showed particular effectiveness under constrained sensing radii.
• Results indicate TransMARL's practicality and scalability for observation-constrained cooperative control.

Conclusions:

• TransMARL offers a robust solution for cooperative control in multi-agent roundup scenarios with limited perception.
• Further research is needed for broader generalization and formal theoretical characterization.