对离散时间线性系统的反向增强学习基于反向最佳控制
Jiashun Huang1, Dengguo Xu1, Yahui Li1
1School of Automation, Guangxi University of Science and Technology, Liuzhou 54500, China.
ISA transactions
|May 23, 2025
概括
本研究引入了新的反强化学习 (IRL) 算法,用于对线性系统重建成本函数. 这些基于模型和部分无模型的方法有效地从专家数据中回收系统成本.
科学领域:
- 控制系统工程 控制系统工程
- 机器学习 机器学习
- 优化优化 优化优化
背景情况:
- 最佳的控制问题需要准确的成本函数来实现有效的系统设计.
- 反向增强学习 (IRL) 旨在从专家的演示中推断出这些成本函数.
- 在许多工程应用中,线性时间不变系统是基本的.
研究的目的:
- 开发新的IRL算法,用于在离散时间线性时间不变系统中重建成本函数.
- 解决成本函数回收的基于模型和部分无模型场景.
- 确保拟议的IRL方法的稳定性和趋同性.
主要方法:
- 一个基于模型的IRL算法是通过重新制定最佳控制增益公式来呈现的.
- 使用辅助控制输入和外内循环结构开发了一个部分无模型的IRL框架.
- 算法包括通过代数里卡蒂方程 (ARE) 更新控制增益,用于成本矩阵校正的梯度下降,以及通过反向最佳控制 (IOC) 更新权重矩阵.
主要成果:
- 建议的算法成功地使用专家代理数据 (状态和输入测量) 重建成本函数.
- 部分无模型方法使成本函数重建成为可能,即使输入矩阵是未知的.
- 理论上证明了算法的融合和闭环系统的稳定性.
结论:
- 开发的IRL算法对于线性系统中的成本函数重建是有效的.
- 这些方法为已知和部分未知的系统模型提供了可靠的解决方案.
- 模拟结果验证了拟议的IRL技术的实际适用性和性能.
相关概念视频
01:26Feedback control systems
442
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
442
01:21Linear Approximation in Time Domain
130
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
130
01:23Linear time-invariant Systems
452
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
452
01:26Linear Approximation in Frequency Domain
142
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
142
01:21First Order Systems
167
First-order systems, such as RC circuits, are foundational in understanding dynamic systems due to their straightforward input-output relationship. Analyzing their responses to different input functions under zero initial conditions reveals significant insights into system behavior.
When a first-order system is subjected to a unit-step input, its response is characterized by its transfer function. By applying the Laplace transform of the unit-step input to the transfer function, expanding the...
When a first-order system is subjected to a unit-step input, its response is characterized by its transfer function. By applying the Laplace transform of the unit-step input to the transfer function, expanding the...
167
01:24PI Controller: Design
526
Proportional Integral (PI) controllers are a fundamental component in modern control systems, widely used to enhance performance and mitigate steady-state errors. They are particularly effective in applications such as automatic brightness adjustment on smartphones, where they excel at mitigating steady-state errors for step-function inputs. Unlike PD controllers, which require time-varying errors to function optimally, PI controllers leverage their integral component to address residual...
526