对于线性离散时间系统的跟踪问题,无模型的Q学习
IEEE transactions on neural networks and learning systems
|February 21, 2024
概括
一个新的无模型的Q学习算法解决了线性系统中未知动态的跟踪错误. 这种方法将追踪转化为监管,确保精确的控制政策趋同,使用最小的数据.
科学领域:
- 控制系统工程 控制系统工程
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 线性离散时间系统中的跟踪问题往往受到未知的系统动态的影响,阻碍了精确的控制.
- 现有的自适应动态编程 (ADP) 和Q学习方法在处理完全未知的系统参数以跟踪任务方面存在局限性.
研究的目的:
- 提出一种无模型的Q学习算法,用于解决线性离散时间系统中的跟踪问题,其动态不明.
- 开发一种新的性能指数,将跟踪问题转化为监管问题,从而消除跟踪错误.
主要方法:
- 引入了一个无模型的Q学习算法,利用了一个增强的性能指数,添加了一个产品术语.
- 控制策略是通过在线系统状态,控制输入和参考轨迹信息来代推断的,没有先前的系统知识.
- 采用了政策之外的方法来优化数据的使用,从而推导出最佳的控制政策.
主要成果:
- 拟议的性能指数有效地将跟踪问题转化为监管问题,旨在实现零稳定状态误差.
- 代的Q学习方法与新的性能指数相结合,产生了一个控制策略,理论上消除了跟踪错误.
- 数字模拟验证了拟议的算法的有效性,以实现准确的系统跟踪.
结论:
- 开发的无模型的Q学习算法为在动态不明的系统中跟踪控制提供了强大的解决方案.
- 新的绩效指数和代更新策略确保准确的跟踪和高效的政策学习.
- 这种方法比现有方法提供了显著的进步,特别是在系统信息有限的场景中.
相关概念视频
01:21Linear Approximation in Time Domain
81
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
81
01:23Linear time-invariant Systems
258
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
258
01:26Feedback control systems
313
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
313
01:26Linear Approximation in Frequency Domain
91
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
91
01:21First Order Systems
91
First-order systems, such as RC circuits, are foundational in understanding dynamic systems due to their straightforward input-output relationship. Analyzing their responses to different input functions under zero initial conditions reveals significant insights into system behavior.
When a first-order system is subjected to a unit-step input, its response is characterized by its transfer function. By applying the Laplace transform of the unit-step input to the transfer function, expanding the...
When a first-order system is subjected to a unit-step input, its response is characterized by its transfer function. By applying the Laplace transform of the unit-step input to the transfer function, expanding the...
91
01:27State Space Representation
208
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
208