Reinforcement learning based proportional-integral-derivative controllers design for consensus of multi-agent systems

Area of Science:

• Control Systems Engineering
• Artificial Intelligence
• Robotics

Background:

• Achieving consensus in multi-agent systems is crucial for coordinated behavior.
• Traditional Proportional-Integral-Derivative (PID) tuning methods often rely on system models or extensive data.
• Existing methods struggle with unknown dynamics and real-time adaptation.

Purpose of the Study:

• To develop a novel, self-learning PID tuning method for multi-agent systems.
• To guarantee optimal consensus and performance optimization for all agents.
• To enable controller parameter updates through active environmental interaction.

Main Methods:

• Formulation of the PID control-based consensus problem for multi-agent systems.
• Conversion of PID gain finding into a nonzero-sum game problem.
• Proposal of an off-policy Q-learning algorithm with a critic-only structure for PID gain updates using only data.

Main Results:

• The proposed method updates PID controller parameters without prior knowledge of agent dynamics.
• Guaranteed consensus and performance optimization are achieved for the agents.
• Simulation results demonstrate the effectiveness of the novel self-learning PID tuning approach.

Conclusions:

• The developed self-learning PID tuning method offers a robust solution for multi-agent consensus.
• This approach overcomes limitations of traditional model-based and data-driven methods.
• The method shows significant potential for applications requiring adaptive and optimal control in multi-agent systems.