A Distributed Multi-Agent Formation Control Method Based on Deep Q Learning

Area of Science:

• Robotics and Control Systems
• Artificial Intelligence
• Multi-Agent Systems

Background:

• Distributed control is crucial for multi-agent systems (MAS) to achieve mission objectives.
• Existing distributed formation control methods often neglect collision risks, limiting their practical application.
• Ensuring safety and efficient formation control in MAS requires addressing inter-agent collision avoidance.

Purpose of the Study:

• To propose a novel distributed formation control method for MAS that explicitly incorporates collision avoidance.
• To develop a practical and easily trainable approach for multi-agent collision avoidance formation control.
• To enhance the reliability and applicability of MAS formation control in complex environments.

Main Methods:

• Decomposition of the MAS formation control problem into pair-wise unit formation problems.
• Application of a deep Q network (DQN) for modeling the unit controller, trained with a reshaped reward function and prioritized experience replay.
• Utilizing min-max fusion of DQN value functions for agents to prioritize responses to the most critical avoidance scenarios.

Main Results:

• The proposed method enables agents to achieve formation goals while effectively avoiding collisions.
• The DQN controller, shared among agents, provides distinct commands based on individual observations.
• Simulations demonstrate the effectiveness of the unit formation and multi-agent formation control strategies.

Conclusions:

• The developed distributed formation control method offers an effective solution for collision avoidance in MAS.
• The DQN-based approach simplifies training and enhances the practicality of multi-agent formation control.
• This research contributes to the advancement of safer and more robust autonomous systems.