对于具有数据失效的离散时间系统的反向增强学习.
IEEE transactions on cybernetics
|March 4, 2025
概括
本研究引入了反向增强学习 (IRL) 算法,使控制系统能够有效地跟踪目标系统,即使在无线传输过程中丢失数据. 这些方法允许系统学习未知的目标行为,以提高跟踪性能.
科学领域:
- 控制系统工程 控制系统工程
- 机器学习 机器学习
- 无线通信无线通信
背景情况:
- 网络控制系统 (NCS) 在无线传输过程中面临数据丢失的挑战.
- 追踪控制需要了解未知的目标系统的行为,通常由未知的成本函数定义.
研究的目的:
- 开发反向强化学习 (IRL) 算法,用于跟踪随机状态中断的线性NCS控制.
- 使一个受控系统能够推断出目标系统的未知成本函数和最佳策略.
主要方法:
- 开发了一个基于模型的IRL算法,集成史密斯预测器进行状态估计.
- 提出了一个状态-dropout-aware反向Q学习算法,只需要可访问的系统数据.
主要成果:
- 拟议的算法有效地推断出目标的成本函数和最佳控制政策.
- 理论有效性被严格确立.
- 数字模拟证实了算法的实际有效性.
结论:
- 开发的IRL算法为随机状态中断的NCS中跟踪控制提供了强大的解决方案.
- 这些方法提高了跟踪性能,使系统能够学习和适应未知的目标动态,尽管通信不确定性.
相关概念视频
01:24Reinforcement Schedules
126
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
126
01:31Classification of Systems-II
133
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
133
01:18Second Order systems II
79
In an underdamped second-order system, where the damping ratio ζ is between 0 and 1, a unit-step input results in a transfer function that, when transformed using the inverse Laplace method, reveals the output response. The output exhibits a damped sinusoidal oscillation, and the difference between the input and output is termed the error signal. This error signal also demonstrates damped oscillatory behavior. Eventually, as the system reaches a steady state, the error diminishes to zero.
79
01:11Sampling Continuous Time Signal
195
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
195
01:26Feedback control systems
268
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
268
01:24BIBO stability of continuous and discrete -time systems
321
System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
321