通过包括直接和间接的途径来增强强化学习模型的增强,可以提高在条形依赖任务上的表现
1Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, Virginia, United States of America.
PLoS computational biology
|August 18, 2023
概括
这项研究介绍了TD2Q,一种新的强化学习模型,更好地模仿基底. TD2Q通过结合直接和间接途径神经元动态来改善动物在学习任务中的行为模拟.
科学领域:
- 神经科学是一个神经科学.
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
背景情况:
- 奖励学习是多巴胺依赖的,涉及皮质-状路径中的突触可塑性.
- 现有的强化学习模型使用奖励预测错误,但难以处理复杂的目标导向行为,如更新和反转.
研究的目的:
- 介绍TD2Q,一种新的强化学习模型,它更好地代表了基底节电路.
- 提高动物学习行为的模拟,并了解直接和间接途径的作用.
主要方法:
- 开发了TD2Q,这是一个两个Q矩阵模型 (G是直接途径神经元,N是间接途径神经元),通过时间差异奖励预测错误进行更新.
- 在多步骤任务中测试了TD2Q,包括灭绝,更新,歧视,奖励概率切换和序列学习.
主要成果:
- TD2Q模拟与选择和序列学习任务中的动物行为密切匹配.
- 时间差异奖励预测错误对于多步任务学习至关重要.
- 阻止N矩阵更新损害了歧视学习,与实验发现保持一致.
- 序列学习性能在双矩阵结构中显著提高.
结论:
- 将基底腺生理纳入强化学习模型可以提高表现和行为模仿.
- TD2Q提供了关于直接和间接条状通路神经元在学习中的功能作用的见解.
相关概念视频
Indirect Motor Pathways
1.6K
The indirect motor or extrapyramidal pathways originate in the brainstem, the lower portion of the brain that connects it to the spinal cord. They consist of several distinct tracts, each with specialized functions. The four main tracts of the indirect motor pathways are the vestibulospinal tract, the reticulospinal tract, the tectospinal tract, and the rubrospinal tract.
The vestibulospinal tract originates in the vestibular nuclei of the brainstem. The vestibular system detects changes in...
The vestibulospinal tract originates in the vestibular nuclei of the brainstem. The vestibular system detects changes in...
1.6K
Long-term Potentiation
55.3K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.3K
Real-World Application of Classical Conditioning
626
Classical conditioning not only includes the initial pairing of stimuli but also extends to more complex forms, such as higher-order conditioning. Higher-order conditioning involves creating associations beyond the primary conditioned stimulus, resulting in a chain of conditioned responses.
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
626
Adrenergic Agonists: Indirect-Acting Agents
1.7K
Indirect-acting adrenergic agonists potentiate the effects of endogenous catecholamines through different mechanisms without directly binding to adrenoceptors.
One mechanism involves depleting stored catecholamines by displacing them from synaptic vesicles. These agents, known as "displacers," are transported into vesicles at the expense of noradrenaline. Examples include amphetamine and tyramine, which lack a catechol moiety, resulting in prolonged action, improved oral...
One mechanism involves depleting stored catecholamines by displacing them from synaptic vesicles. These agents, known as "displacers," are transported into vesicles at the expense of noradrenaline. Examples include amphetamine and tyramine, which lack a catechol moiety, resulting in prolonged action, improved oral...
1.7K
Cognitive Learning
421
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
421
Direct Motor Pathways
2.1K
The direct motor pathways, also known as the pyramidal tracts, are a group of neural pathways that originate in the brain and descend through the spinal cord. They control the voluntary movement of the body. There are two major direct motor pathways: the corticospinal and the corticobulbar tracts.
The corticospinal tract is responsible for the voluntary movement of the limbs and trunk. It originates in the cerebral cortex of the brain and descends through the cerebrum's internal capsule and...
The corticospinal tract is responsible for the voluntary movement of the limbs and trunk. It originates in the cerebral cortex of the brain and descends through the cerebrum's internal capsule and...
2.1K


