使用强化学习来控制布尔控制网络的可访问性,以最低成本的状态翻转控制
IEEE transactions on cybernetics
|September 17, 2024
概括
本研究介绍了布尔控制网络 (BCN) 的无模型强化学习. 它开发了最低成本的状态翻转控制方法,优化可达性和减少控制行动.
科学领域:
- 控制理论 控制理论
- 人工智能的人工智能
- 计算机科学 计算机科学
背景情况:
- 布尔控制网络 (BCN) 广泛用于复杂系统的建模.
- 控制BCN以有效地实现所需状态是一个重大挑战.
- 现有的方法经常与大规模网络和优化控制成本作斗争.
研究的目的:
- 提出无模型的强化学习方法,用于BCN中最低成本的状态翻转控制.
- 为了确定最小的翻转集 (翻转内核),确保状态可访问性.
- 为了获得最佳的政策,尽量减少翻转行动的可达性.
主要方法:
- 利用Q学习来确定可访问性和最佳的政策导出.
- 整合转移学习和专门的初始状态以加快Q学习的融合.
- 开发一个基于NCB特征的奖励计划,以实现最佳控制.
- 对于大规模的BCN,采用小内存的Q学习,内存使用上限.
- 引入适应变量奖励,以加速大规模BCN的融合.
主要成果:
- 证明了Q学习在找到翻转内核和最佳控制策略方面的有效性.
- 在小型和大型BCN上验证了拟议的方法.
- 展示了减少内存使用和加速大规模网络的融合.
- 证明了基于NCB特征的奖励计划的最佳性.
结论:
- 拟议的无模型强化学习方法为BCN中最低成本的状态翻转控制提供了有效的解决方案.
- 这些技术增强了可访问性分析和政策优化,特别是在大型系统中.
- 该研究为解决 BCN 控制中的计算挑战提供了新的方法.
相关概念视频
01:17Open and closed-loop control systems
678
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
678
01:21Control System Problem
110
In an open-loop system, such as a basic thermostat, the poles of the transfer function influence the system's response but do not determine its stability. However, when feedback is introduced to form a closed-loop system, such as an advanced thermostat that adjusts heating based on room temperature, stability is governed by the new poles of the closed-loop transfer function.
When forming a closed-loop system, issues can arise if the poles cross into the unstable region, leading to potential...
When forming a closed-loop system, issues can arise if the poles cross into the unstable region, leading to potential...
110
01:21State Space to Transfer Function
179
The conversion of state-space representation to a transfer function is a fundamental process in system analysis. It provides a method for transitioning from a time-domain description to a frequency-domain representation, which is crucial for simplifying the analysis and design of control systems.
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
179
01:23Transfer Function to State Space
206
State-space representation is a powerful tool for simulating physical systems on digital computers, necessitating the conversion of the transfer function into state-space form. Consider an nth-order linear differential equation with constant coefficients, like those encountered in an RLC circuit. The state variables are selected as the output and its n−1 derivatives. Differentiating these variables and substituting them back into the original equation produces the state equations.
In an...
In an...
206
01:22Block Diagram Reduction
167
The process of deriving the transfer function of a control system often involves reducing its block diagram to a single block. This simplification can be achieved through a series of strategic operations, including relocating branch points and comparators. These operations preserve the overall function of the system while allowing for easier manipulation and combination of blocks.
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
167
01:27State Space Representation
171
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
171