适应性强化学习由前带皮质和条纹体因果支持
Robert Louis Treuting1, Kianoush Banaie Boroujeni2, Charles Grimes Gerrity3
1Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37240, USA; Vanderbilt Brain Institute, Nashville, TN 372404, USA.
Neuron
|June 11, 2025
概括
大脑中的自适应强化学习策略至关重要. 前带带皮层 (ACC) 刺激在不确定性下损害了学习,而条纹体刺激改善了它,优化了探索和价值更新.
科学领域:
- 神经科学是一个神经科学.
- 认知科学 认知科学
- 计算神经科学是一种神经科学.
背景情况:
- 强化学习从适应性策略中获益.
- 这些策略包括调整探索-利用,工作记忆和注意力.
- 前带状皮质 (ACC) 和条纹体是关键的大脑区域.
研究的目的:
- 调查ACC和条纹体如何支持适应性学习.
- 检查基于特征的注意力学习中的角色,不确定性和突出性各不相同.
- 确定直接大脑刺激对这些过程的影响.
主要方法:
- 在气体条件下对ACC和条纹体进行电刺激.
- 适应性强化学习的行为测试.
- 计算建模用于解释神经机制.
- 对ACC和条纹体中神经元选择性的分析.
主要成果:
- 在高不确定性情况下,ACC刺激会影响学习;条纹体刺激会改善.
- 建模表明ACC刺激阻碍了勘探优化和预测错误的使用.
- 状体刺激增强了价值预期的更新.
- 在ACC中神经元活动跟踪错误历史和不确定性;条状神经元偏好某些高价值选择.
结论:
- 在适应性强化学习中,ACC和条纹体扮演着不同的,对立的角色.
- 这些区域在不确定性期间优化了对奖励相关线索的探索指导.
- 研究结果提供了对注意力和学习的神经机制的洞察.
相关概念视频
Cognitive Learning
229
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
229
Role of Cerebellum and Prefrontal Cortex in Memory
387
The cerebellum, while traditionally associated with motor control, also plays a crucial role in memory, particularly in procedural memory, which involves learning motor tasks that become automatic through repetition. For example, studies have shown that when the cerebellum is damaged, individuals or animals lose the ability to learn conditioned motor responses, such as the conditioned eye-blink response in classical conditioning experiments with rabbits. This study demonstrates the...
387
Instinctive Drift
193
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
193
Timing and Consequences on Behavior
84
In operant conditioning, the timing of reinforcement is crucial. For animals like rats and cats, immediate reinforcement (within a few seconds) is much more effective than delayed reinforcement. For example, a food reward for a rat needs to follow within 30 seconds of pressing a bar to be effective.
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
84
Association Areas of the Cortex
5.1K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.1K
Associative Learning
309
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
309


