用数据驱动的最佳双边共识控制,通过政策梯度增强学习来控制二级多代理系统
IEEE transactions on cybernetics
|June 12, 2023
概括
本研究介绍了一种基于数据的最佳双方共识控制 (OBCC) 策略,用于未知多代理系统 (MAS). 该方法使用强化学习来实现代理状态的共识,即使具有不同的计算能力.
科学领域:
- 控制系统工程 控制系统工程
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
背景情况:
- 多代理系统 (MAS) 在实现协调行为方面存在挑战,特别是在未知动态的情况下.
- 双边共识控制对于具有合作和竞争互动的系统至关重要.
- 现有的控制策略经常与异步更新和不同的代理能力作斗争.
研究的目的:
- 为了解决未知二级离散时间MAS的最佳双边共识控制 (OBCC) 问题.
- 制定数据驱动的分布式控制策略,确保双方达成共识.
- 设计一种异步控制算法,能够处理代理之间的异质计算能力.
主要方法:
- 建立一个合作竞争网络,以模拟代理互动.
- 分布式政策梯度强化学习 (RL) 的应用,以获得最佳的控制策略.
- 利用实时生成的离线数据集,提高学习效率.
- 使用演员-关键神经网络结构的实现.
- 稳定性和收性分析使用功能分析和利亚普诺夫理论.
主要成果:
- 成功获得了数据驱动的分布式最佳控制策略.
- 拟议的战略保证了对代理人的位置和速度状态的双边共识.
- 异步算法有效地解决了MAS内部的计算异质性.
- 严格分析了MAS的稳定性和学习过程的融合.
结论:
- 开发的基于强化学习的OBCC策略对未知的离散时间MAS有效.
- 异步和数据驱动的方法确保了强大的双边共识,尽管代理的异质性.
- 数字模拟验证了拟议方法的有效性和性能.
相关概念视频
01:18Second Order systems II
137
In an underdamped second-order system, where the damping ratio ζ is between 0 and 1, a unit-step input results in a transfer function that, when transformed using the inverse Laplace method, reveals the output response. The output exhibits a damped sinusoidal oscillation, and the difference between the input and output is termed the error signal. This error signal also demonstrates damped oscillatory behavior. Eventually, as the system reaches a steady state, the error diminishes to zero.
137
01:20Second Order systems I
191
A servo system exemplifies a second-order system, featuring a proportional controller and load elements that ensure the output position aligns with the input position. The relationship between these components is described by a second-order differential equation. Applying the Laplace transform under zero initial conditions yields the transfer function, showing how inputs are converted to outputs in the system.
By reinterpreting the system, one can derive the closed-loop transfer function, which...
By reinterpreting the system, one can derive the closed-loop transfer function, which...
191
01:22Multi-input and Multi-variable systems
133
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
133
01:26Feedback control systems
352
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
352
01:07Time-Domain Interpretation of PD Control
147
Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...
Consider the example of control of motor torque. Initially, a positive...
147
01:09Decision Making: P-value Method
5.5K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.5K