Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning.

Area of Science:

• Robotics and Control Systems
• Artificial Intelligence
• Machine Learning

Background:

• Optimal tracking control is crucial for multiagent systems (MASs).
• Existing reinforcement learning (RL) methods face challenges with unknown nonlinear dynamics in MASs.
• Improving long-term information reception in RL agents is key for enhanced control performance.

Purpose of the Study:

• To develop a novel RL method for optimal tracking control of unknown nonlinear MASs.
• To introduce an internal reinforce reward (IRR) function to enhance agent learning.
• To establish a data-driven, online learning framework for distributed control.

Main Methods:

• Proposed an internal reinforce Q-learning (IrQ-L) method.
• Defined a Q-function based on the IRR function.
• Developed an iterative IrQL algorithm with convergence and stability analysis.
• Implemented a reinforce-critic-actor neural network framework for function estimation.
• Designed a data-driven approach avoiding system dynamics knowledge.

Main Results:

• The proposed IrQ-L method effectively solves the optimal tracking control problem.
• The IRR function improved agents' capability to receive long-term environmental information.
• The reinforce-critic-actor framework successfully estimated IRR, Q-functions, and control schemes.
• Simulations demonstrated superior performance compared to classical methods.

Conclusions:

• The novel IrQ-L method offers an effective solution for optimal tracking control in unknown nonlinear MASs.
• The data-driven, distributed online learning framework enables efficient control without system identification.
• The approach shows significant potential for advancing autonomous multiagent coordination.