在初始稳定OPFB政策下,基于政策代的线性连续时间系统的学习设计
IEEE transactions on cybernetics
|July 22, 2024
概括
本研究为输出反系统 (OPFB) 引入了一种新的政策代 (PI) 方法,克服了初始完全状态反 (FSFB) 政策的局限性. 新方法有效地从OPFB政策中直接学习最佳控制规律.
科学领域:
- 强化学习是一种强化学习.
- 控制理论 控制理论
- 连续时间系统 连续时间系统
背景情况:
- 政策代 (PI) 对于在未知的环境中学习决策规律是有价值的.
- 现有的输出反 (OPFB) 连续时间系统的PI方法需要初始稳定全状态反 (FSFB) 政策,违反了OPFB原则.
- 这种限制阻碍了在只有输出反可用的场景中直接应用PI.
研究的目的:
- 在连续时间系统的初始稳定输出反 (OPFB) 政策下建立政策代 (PI).
- 解决现有的基于PI的控制方法中违反OPFB原则的问题.
- 开发一种高效的PI算法,仅使用OPFB信息来接近最佳控制.
主要方法:
- 使用非政策贝尔曼方程,将任何OPFB政策转化为FSFB政策.
- 传统的PI算法通过在此转换基础上进行额外的代来修改.
- 使用理论分析和案例研究来证明方法的有效性.
主要成果:
- 拟议的方法成功地在初始稳定OPFB政策下建立了政策代 (PI).
- 非策略贝尔曼方程的转换属性使OPFB策略的使用成为可能.
- 修订后的PI算法在OPFB约束下有效地接近了最佳控制规律.
结论:
- 开发的PI方法有效地克服了对连续时间系统的FSFB初始政策的依赖.
- 这项工作为OPFB环境中的PI提供了理论基础和实践演示.
- 拟议的方法增强了强化学习在现实世界系统中控制的适用性,反有限.
更多相关视频
06:45Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
Published on: October 28, 2022
1.6K
08:35Interactive and Visualized Online Experimentation System for Engineering Education and Research
Published on: November 24, 2021
2.4K
相关概念视频
01:26Feedback control systems
303
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
303
01:21Linear Approximation in Time Domain
81
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
81
01:24BIBO stability of continuous and discrete -time systems
381
System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
381
01:21Control System Problem
110
In an open-loop system, such as a basic thermostat, the poles of the transfer function influence the system's response but do not determine its stability. However, when feedback is introduced to form a closed-loop system, such as an advanced thermostat that adjusts heating based on room temperature, stability is governed by the new poles of the closed-loop transfer function.
When forming a closed-loop system, issues can arise if the poles cross into the unstable region, leading to potential...
When forming a closed-loop system, issues can arise if the poles cross into the unstable region, leading to potential...
110
01:23Linear time-invariant Systems
242
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
242
01:24Pole and System Stability
268
The transfer function is a fundamental concept representing the ratio of two polynomials. The numerator and denominator encapsulate the system's dynamics. The zeros and poles of this transfer function are critical in determining the system's behavior and stability.
Simple poles are unique roots of the denominator polynomial. Each simple pole corresponds to a distinct solution to the system's characteristic equation, typically resulting in exponential decay terms in the system's...
Simple poles are unique roots of the denominator polynomial. Each simple pole corresponds to a distinct solution to the system's characteristic equation, typically resulting in exponential decay terms in the system's...
268