当模型重要时:环境需求指导基于模型和无模型控制之间的仲裁.
Leslie K Held1, Elise Lesage2, Wouter Kool3
1Department of Experimental Psychology, Ghent University, Ghent, Belgium. leslie.held@ugent.be.
Cognitive, affective & behavioral neuroscience
|October 13, 2025
概括
人们可以学习在基于环境需求的无模型和基于模型的控制策略之间切换. 适应状态交替或重复影响我们是否重复行动或提前计划.
科学领域:
- 认知心理学 认知心理学
- 神经科学是一个神经科学.
- 强化学习是一种强化学习.
背景情况:
- 人类的决策包括习惯性 (无模型) 和目标导向 (基于模型) 控制.
- 已知使用策略的个体间差异,但对适应能力的理解较少.
研究的目的:
- 调查人类是否可以学会根据环境需求调节无模型与基于模型的控制.
- 为了确定参与者是否根据环境状态交替或重复的频率来调整他们的策略.
主要方法:
- 140名参与者进行了两阶段的决策任务.
- 通过改变第一阶段状态交替与重复的频率来操纵环境需求.
- 无模型行为是通过重复奖励的选择来表示的;基于模型的行为是通过在各州普遍化奖励来表示的.
主要成果:
- 暴露于更多第一阶段状态交替的参与者在随后的测试阶段表现出更多基于模型的控制.
- 相反,暴露于更多第一阶段状态重复的参与者表现出更多的无模型行为.
- 这表明了战略之间的学习仲裁.
结论:
- 人类可以根据不断变化的环境需求来调整其强化学习策略 (无模型与基于模型).
- 这种适应性似乎是一个对环境敏感的成本效益分析.
- 调查结果强调了决策控制的动态性质.
相关概念视频
Feedback control systems
685
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
685
Open and closed-loop control systems
1.6K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.6K
Controller Configurations
352
Controller configurations are crucial in a car's cruise control system because they manage speed over time to maintain a consistent pace regardless of road conditions, thereby meeting design goals. In traditional control systems, fixed-configuration design involves predetermined controller placement. System performance modifications are known as compensation.
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
Control-system compensation involves various configurations, most commonly series or cascade compensation, in which the controller...
352
Control Systems
1.8K
Control systems are everywhere in contemporary society, influencing diverse applications from aerospace to automated manufacturing. These systems can be found naturally within biological processes, such as blood sugar regulation and heart rate adjustment in response to stress, as well as in man-made systems like elevators and automated vehicles. A control system is essentially a network of subsystems and processes that collaboratively convert specific inputs into desired outputs.
At the heart...
At the heart...
1.8K
Time-Domain Interpretation of PD Control
371
Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...
Consider the example of control of motor torque. Initially, a positive...
371
PD Controller: Design
622
In automotive engineering, car suspension systems often employ Proportional Derivative (PD) controllers to enhance performance. PD controllers are utilized to adjust the damping force in response to road conditions. A controller, acting as an amplifier with a constant gain, demonstrates proportional control, with output directly mirroring input.
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
Designing a continuous-data controller requires selecting and linking components like adders and integrators, which are fundamental in Proportional,...
622


