通过Q学习方法对离散时间多代理系统的合作输出调节的无模型算法.
IEEE transactions on cybernetics
|March 27, 2025
概括
一个新的无模型的Q学习算法使得多个代理系统的合作输出调节具有未知的参数. 这种数据驱动的方法确保了政策稳定性,并避免了系统模型要求.
科学领域:
- 控制系统工程 控制系统工程
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
背景情况:
- 合作输出监管对于多代理系统至关重要.
- 在实际应用中,未知的系统参数构成了重大挑战.
- 现有的方法通常需要完整的系统模型,从而限制了它们的适用性.
研究的目的:
- 开发一种无模型的Q学习算法,用于在离散时间多代理系统中的合作输出调节.
- 为了应对未知的系统参数的挑战.
- 确保政策稳定性和学习算法的融合.
主要方法:
- 提出了一个无模型的Q学习算法,独立于系统参数运行.
- 立即成本的制定消除了解决监管方程的需要.
- 引入了一个数据驱动的算法,用于计算不稳定的初始策略的初始稳定收益.
- 算法代的稳定性和独特的Q函数矩阵条件是正式衍生出来的.
主要成果:
- 拟议的Q学习算法实现了简化结构,直接确定最佳政策.
- 正式稳定性分析证实了每个算法代的趋同.
- 数据驱动的方法成功地确保了趋同到稳定,即使在不稳定的初始政策.
- 证明分布式观察者和激发噪声不会引入偏差.
结论:
- 无模型的Q学习方法为在具有未知参数的多代理系统中进行合作输出调节提供了有效的解决方案.
- 开发的算法确保了稳定性和趋同性,提高了实际应用性.
- 模拟示例验证了拟议方法的有效性和稳定性.
相关概念视频
Open and closed-loop control systems
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal and...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal and...
Multi-input and Multi-variable systems
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Parameters Affecting Nonlinear Elimination: Zero-Order Input, First-Order Absorption and Two-Compartment Model
Drugs administered through various routes can lead to nonlinear elimination, resulting in complex pharmacokinetic behaviors crucial to understanding efficacious drug dosing.
When a drug is administered through a constant intravenous infusion and eliminated via nonlinear pharmacokinetics, it follows zero-order input. For example, oral drugs undergo first-order absorption upon administration and are eliminated through nonlinear pharmacokinetics.
In the case of subcutaneously administered drugs,...
When a drug is administered through a constant intravenous infusion and eliminated via nonlinear pharmacokinetics, it follows zero-order input. For example, oral drugs undergo first-order absorption upon administration and are eliminated through nonlinear pharmacokinetics.
In the case of subcutaneously administered drugs,...
Modeling with Differential Equations
Population dynamics can be described mathematically by considering the population size P(t) as a function of time. The rate of change of the population is then represented by the derivative of P(t). A simple assumption is that the rate of growth is proportional to the size of the population itself. This leads to an exponential growth model, where the population increases rapidly without bound. While this is a useful first approximation, it does not reflect realistic long-term...
Lagrange Multipliers: Problem Solving
A silo with a cylindrical base, flat bottom, and hemispherical roof is a common design in agricultural and industrial storage due to its structural efficiency and ease of construction. Optimizing its dimensions to maximize storage capacity for a given amount of material—i.e., a fixed surface area—is a classic problem in applied calculus and engineering design. The key parameters are the radius r of the base and the height h of the cylindrical section.The total volume of the silo is obtained by...


