对时空系统的基于强化学习的估计
Saviz Mowlavi1, Mouhacine Benosman2
1Mitsubishi Electric Research Laboratories, Cambridge, MA, 02139, USA. mowlavi@merl.com.
Scientific reports
|September 28, 2024
概括
本研究为动态系统引入了一种新的强化学习减少顺序估计器 (RL-ROE). RL-ROE有效地弥补模型错误,即使在有限的传感器数据下,也可以准确地估计状态.
科学领域:
- 动态系统估计的估计.
- 科学机器学习科学机器学习
- 部分微分方程部分微分方程.
背景情况:
- 像卡尔曼过器这样的状态估计器对于动态系统至关重要,通常使用减少顺序模型 (ROM) 进行计算效率.
- ROM可以引入重大错误,降低传统状态估计器的性能.
- 通过部分微分方程 (PDEs) 规范的时空系统的准确状态估计仍然是一个挑战,特别是在稀疏的传感器数据的情况下.
研究的目的:
- 开发一种新的状态估计器,克服时空系统中ROM的局限性.
- 引入强化学习减少顺序估计器 (RL-ROE),利用强化学习来纠正ROM不准确性.
- 为了实现由参数PDEs控制的复杂系统的准确实时传感.
主要方法:
- 通过将通过强化学习训练的非线性策略纳入估计器的校正项中,开发了一个强化学习减少顺序估计器 (RL-ROE).
- 将RL-ROE应用于由伯格斯和纳维尔-斯托克斯方程控制的时空系统,并纳入参数不确定性.
- 在稀疏的传感器测量条件下,使用相同的ROM对传统卡尔曼波器进行RL-ROE的性能评估.
主要成果:
- 经过训练的RL-ROE与基于相同ROM的卡尔曼波器相比,表现优异,特别是在使用很少传感器的场景中.
- RL-ROE实现了对高维状态的准确即时估计,有效地处理未知的初始条件和物理参数值.
- 在RL-ROE内部的非线性政策成功地弥补了ROM错误,同时利用了不完美的动态知识.
结论:
- 强化学习减少顺序估计器 (RL-ROE) 提供了一种强大的方法来提高对具有参数 PDE 的系统的状态估计精度.
- RL-ROE能够有效地弥补减少顺序模型错误,从而比传统方法提高性能.
- 这种方法为复杂的动态系统的轻量级实时传感应用铺平了道路.
相关概念视频
01:21Linear Approximation in Time Domain
69
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
69
01:27State Space Representation
169
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
169
01:22Multi-input and Multi-variable systems
101
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
101
01:24Reinforcement Schedules
135
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
135
01:05Estimation of the Physical Quantities
4.2K
On many occasions, physicists, other scientists, and engineers need to make estimates of a particular quantity. These are sometimes referred to as guesstimates, order-of-magnitude approximations, back-of-the-envelope calculations, or Fermi calculations. The physicist Enrico Fermi was famous for his ability to estimate various kinds of data with surprising precision. Estimating does not mean guessing a number or a formula at random. Instead, estimation means using prior experience and sound...
4.2K
01:31Classification of Systems-II
136
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
136