一种不准确的顺序二次编程方法,用于学习和控制反复的神经网络
IEEE transactions on neural networks and learning systems
|January 31, 2024
概括
本研究引入了一种用于部分可观测的马尔科夫决策过程 (POMDPs) 的新两阶段方法. 它使用循环神经网络 (RNN) 进行系统识别和前神经网络 (FNN) 进行控制,增强决策能力.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 控制理论 控制理论
背景情况:
- 部分可观测的马尔科夫决策过程 (POMDP) 在建模和控制方面存在重大挑战,原因是隐藏的状态.
- 现有的方法经常在复杂系统的可扩展性和计算效率方面扎.
研究的目的:
- 开发一种有效的两阶段方法来解决POMDPs.
- 为POMDPs的识别和控制阶段引入一个新的框架.
主要方法:
- 在识别阶段,用于反复神经网络学习 (iSQPRL) 的不准确的顺序二次编程框架,将系统与动态一致的超射反复神经网络 (DCRNN) 进行近似.
- 学习问题是作为一个受约束的优化问题,用一个重新启动的克里洛夫子空间代方案分析二次编程 (QP) 子问题.
- 一个通用的高斯-牛顿 (GGN) 算法被适用于控制阶段,在RNN模型之上设计了一个前神经网络 (FNN) 控制器.
主要成果:
- 在POMDP识别阶段,iSQPRL框架在接近系统动态方面表现出有效性.
- GGN算法促进了FNN控制器的高效设计,利用曲率项的近似值和调整函数属性来进行小批次步骤大小选择.
- 模拟结果验证了整体方法,表明其在解决POMDPs方面的有效性.
结论:
- 拟议的两阶段方法为POMDPs提供了一个强大而有效的解决方案.
- 用于识别的iSQPRL和用于控制的GGN的集成代表了在部分可观测性下处理复杂的顺序决策问题的重大进展.
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
55
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
55
Linear Approximation in Frequency Domain
91
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
91
State Space Representation
208
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
208
Second Order systems II
113
In an underdamped second-order system, where the damping ratio ζ is between 0 and 1, a unit-step input results in a transfer function that, when transformed using the inverse Laplace method, reveals the output response. The output exhibits a damped sinusoidal oscillation, and the difference between the input and output is termed the error signal. This error signal also demonstrates damped oscillatory behavior. Eventually, as the system reaches a steady state, the error diminishes to zero.
113
Root-Locus Method
151
A cruise control system in a car is designed to maintain a specified speed automatically by adjusting the gas pedal. The system continuously measures the vehicle's speed and makes fine adjustments to the pedal to achieve this goal. The root locus method is particularly useful for understanding how the cruise control system's behavior changes under varying conditions, such as when the car goes uphill, downhill, or faces strong wind resistance.
This system can be represented by a block...
This system can be represented by a block...
151
Feedback control systems
314
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
314


