通过轨道前皮层进行超强化学习
Ryoma Hattori1,2,3,4,5, Nathan G Hedrick6,7,8,9, Anant Jain10
1Department of Neurobiology, University of California San Diego, La Jolla, CA, USA. rhattori0204@gmail.com.
Nature neuroscience
|November 13, 2023
概括
轨道前皮层 (OFC) 在大脑中调解元强化学习 (meta-RL). 在OFC中,突触可塑性对meta-learning至关重要,使适应性决策成为可能.
科学领域:
- 神经科学是一个神经科学.
- 计算神经科学是一种神经科学.
- 认知科学 认知科学
背景情况:
- 超强化学习 (meta-RL) 在深度RL模型中增强了概括性.
- 前额叶皮质被假设为中介基于大脑的meta-RL,但证据有限.
研究的目的:
- 调查轨道前皮层 (OFC) 在调解元RL中的作用.
- 阐明 OFC 中 RL 算法的神经机制和时间尺度.
主要方法:
- 训练小鼠和深度RL模型在概率逆转学习任务上.
- 研究了OFC中Ca2+/calmodulin-dependent蛋白激酶II-dependent突触可塑性的必要性,以实现元学习.
- 监测了OFC活动,并评估了OFC无活化的行为影响.
主要成果:
- OFC介导meta-RL,突触可塑性对meta-learning至关重要,而不是专家在会话RL中的专家.
- 在后期的meta-learning中,OFC活动强有力的编码了价值信号,而OFC失活会影响RL行为.
- 纵向跟踪显示,元学习逐渐塑造OFC人口价值编码以指导行为.
结论:
- OFC实现了meta-RL,使用了不同的神经机制和时间表.
- 在OFC内共存的两个RL算法支持自适应性决策.
相关概念视频
Operant Conditioning Intervention
60
Operant conditioning serves as a foundational principle in therapeutic interventions aimed at modifying maladaptive behaviors. Central to this approach is the notion that behaviors, both adaptive and maladaptive, are learned through reinforcement. By analyzing the environmental factors that reinforce problematic behaviors, clinicians can design interventions to weaken these reinforcements and replace maladaptive behaviors with healthier alternatives.
In operant conditioning, behaviors that are...
In operant conditioning, behaviors that are...
60
Association Areas of the Cortex
5.4K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.4K
Operant Conditioning
1.6K
Operant conditioning, a key concept in behavioral psychology, involves using reinforcement and punishment to alter the likelihood of a behavior being repeated. B.F. introduced this type of conditioning. Skinner focused on voluntary behaviors and the consequences that follow them, influencing whether these behaviors will be strengthened or diminished.
Reinforcement in operant conditioning can be positive or negative, both of which serve to increase the likelihood of a behavior. Positive...
Reinforcement in operant conditioning can be positive or negative, both of which serve to increase the likelihood of a behavior. Positive...
1.6K
Law of Effect
1.4K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
1.4K
Cognitive Learning
247
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
247


