在部分可观测环境中对混乱系统控制进行强化学习.
Max Weissenbacher1,2, Anastasia Borovykh1, Georgios Rigas2
1Department of Mathematics, Imperial College London, London, SW7 2AZ UK.
概括
用有限的信息控制混乱的系统是具有挑战性的. 基于注意力的强化学习框架,使用变压器,在混乱的流体动力学中显著提高了控制性能,即使传感器较少.
科学领域:
- 流体动力学 流体动力学
- 控制理论 控制理论 控制理论
- 机器学习 机器学习
背景情况:
- 控制混乱系统在工程中至关重要,但由于传感有限,现实应用面临部分可观测性.
- 与完全可观测性相比,部分可观测性降低了控制性能.
- 记忆类型对混乱状态下控制器性能的影响仍然不太清楚.
研究的目的:
- 通过部分观察,研究强化学习来控制混乱的流动.
- 评估性能损失与降低传感器可用性.
- 将循环神经网络 (RNN) 与基于变压器的新型记忆机制进行比较.
主要方法:
- 利用Kuramoto-Sivashinsky方程与强迫作为一个模型系统.
- 在各种动态状态下测试控制,从轻微到强烈的混乱.
- 实现并比较基于RNN的内存与基于变压器的注意力机制.
主要成果:
- 随着传感器数量的减少,性能恶化被量化.
- 基于注意力的变压器框架在不同的混乱制度中表现出强的超越性.
- 这种新的机制在高度混乱的环境中显示出更好的控制.
结论:
- 基于注意力的机制,特别是变压器,非常适合控制混乱系统.
- 这种方法在具有挑战性的,高度混乱的流体动力学中提供了增强的控制.
- 这些发现推动了人工智能在复杂动态系统控制中的应用.
相关概念视频
Observational Learning
832
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
832
Feedback control systems
685
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
685
Open and closed-loop control systems
1.6K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.6K
Control Systems
1.8K
Control systems are everywhere in contemporary society, influencing diverse applications from aerospace to automated manufacturing. These systems can be found naturally within biological processes, such as blood sugar regulation and heart rate adjustment in response to stress, as well as in man-made systems like elevators and automated vehicles. A control system is essentially a network of subsystems and processes that collaboratively convert specific inputs into desired outputs.
At the heart...
At the heart...
1.8K
Avoidance Learning and Learned Helplessness
2.5K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.5K
Multi-input and Multi-variable systems
389
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
389


