在不确定动态的非线性系统的双级最佳控制中使用LSTM授权的强化学习
Roya Khalili Amirabadi1, Mohsen Jalaeian-Farimani2, Omid S Fard1
1Department of Applied Mathematics, Ferdowsi University of Mashhad, Mashhad, Iran.
ISA transactions
|November 27, 2025
概括
本研究提出了一个新的双层优化框架,用于对非线性系统进行强有力的控制. 它使用长期短期记忆网络和强化学习来在不确定的环境中适应性轨迹跟踪.
科学领域:
- 机器人和控制系统 机器人和控制系统
- 人工智能的人工智能
- 非线性系统动态 非线性系统动态
背景情况:
- 自主系统需要强大的控制策略,以便在动态和不确定的环境中可靠运行.
- 传统的控制方法经常与时间变化的干扰和复杂的非线性动态作斗争.
- 强化学习 (RL) 和深度学习为自适应控制提供了有希望的途径,但需要仔细的整合和稳定性保证.
研究的目的:
- 开发一种新的双级优化框架,以优化对具有不确定的动态的非线性连续时间系统的最佳控制.
- 将长短期记忆 (LSTM) 网络与关键演员强化学习 (RL) 架构集成,以增强自适应控制.
- 为了实现强大的轨迹跟踪,而不需要线下培训,确保适应时间变化的干扰.
主要方法:
- 一个双层优化框架,将基于哈密尔顿的最佳控制与在线不确定性估计相结合.
- 使用一个演员关键RL架构,其中主级优化控制策略 (灵感来自HJB),奴隶级使用LSTM网络进行动态不确定性估计.
- 实施严格的稳定性分析,以证明追踪错误的统一的最终边界性.
主要成果:
- 与传统的自适应控制和基于模型的虚拟参考轨迹方案相比,证明了优越的跟踪精度,能源效率和干扰排斥能力.
- 通过广泛的模拟,验证了该框架的有效性,该模拟用于执行各种轨迹的滑行方向盘追踪机器人.
- 证实了框架能够处理时间变化的干扰并适应不确定的系统动态的能力.
结论:
- 拟议的双级优化框架为具有不确定性的非线性系统的最佳控制提供了一个计算效率高且理论上有基础的解决方案.
- 这种方法推进了基于RL的最佳控制范式,为在不可预测环境中运行的自主系统提供了可扩展的解决方案.
- 集成的LSTM网络和关键参与者RL确保了强大的和适应性轨迹跟踪,提高系统性能和可靠性.
相关概念视频
01:23Linear time-invariant Systems
846
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
846
01:21Linear Approximation in Time Domain
323
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
323
01:26Feedback control systems
663
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
663
01:17Open and closed-loop control systems
1.5K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.5K
01:27State Space Representation
502
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
502
01:10Control Systems
1.8K
Control systems are everywhere in contemporary society, influencing diverse applications from aerospace to automated manufacturing. These systems can be found naturally within biological processes, such as blood sugar regulation and heart rate adjustment in response to stress, as well as in man-made systems like elevators and automated vehicles. A control system is essentially a network of subsystems and processes that collaboratively convert specific inputs into desired outputs.
At the heart...
At the heart...
1.8K