一个强化学习模型,用于逐步比率计划的选择轨迹
Keiko Ihara1, Yu Shikano1,2, Sae Kato1
1Division of Brain Sciences, Institute for Advanced Medical Research, Keio University School of Medicine, Tokyo, Japan.
Frontiers in behavioral neuroscience
|January 25, 2024
概括
一个新的坚持模式通过分析选择行为,而不仅仅是断点来更好地捕捉杆压力任务中的动机. 这个模型准确地预测了多巴胺变化和药物对动机的影响.
科学领域:
- 神经科学是一个神经科学.
- 行为科学 行为科学
- 计算精神病学是一种计算精神病学.
背景情况:
- 渐进式比率 (PR) 杆按压任务是标准的动机度量.
- 当前的公关任务只提供一个突破点指标,限制了对动机变化的分析.
- 在公关任务期间,杆按压和杂志鼻子的变化没有被很好地捕捉到.
研究的目的:
- 开发先进的强化学习模型,用于分析PR任务中的选择行为.
- 确定一个模型,可以捕捉超越断点的动机动态.
- 用多巴胺测量来研究动机的神经相关性.
主要方法:
- 构建了四个强化学习模型,包括Q学习和坚持不的模型.
- 嵌入增强杆压力机,非增强杆压力机,以及杂志鼻孔插入模型.
- 使用纤维光度与多巴胺传感器 (GRABDA2m) 来测量PR任务期间雄性小鼠的多巴胺.
- 服用甲基胺 (METH) 来评估模型捕捉药物效应的能力.
主要成果:
- 力模型最好预测行为变化,包括在杂志鼻后减少奖励预测错误 (RPEs).
- 腹腔条纹体中的多巴胺动态与模型的预测相关.
- 甲基胺增加了杂志的鼻刺,但没有改变断点,这种调节被坚持模型捕获.
- 该模型强调了初始动作值在行为调制中的作用.
结论:
- 坚持模型提供了更细致的理解 PR 任务的动机和选择行为.
- 这种模型可以揭示精神活性药物对动机过程的影响.
- 这些发现支持使用计算模型来分析复杂的行为和神经信号.
相关概念视频
Reinforcement Schedules
148
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
148
Role of Shaping in Operant Conditioning
322
Shaping is a technique used in operant conditioning to train complex behaviors by rewarding successive approximations toward the target behavior. This method is necessary because organisms are unlikely to perform complex behaviors spontaneously. Instead, shaping breaks down the desired behavior into small, manageable steps.
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
322
Law of Effect
1.4K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
1.4K
Reinforcement
209
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
209
Timing and Consequences on Behavior
94
In operant conditioning, the timing of reinforcement is crucial. For animals like rats and cats, immediate reinforcement (within a few seconds) is much more effective than delayed reinforcement. For example, a food reward for a rat needs to follow within 30 seconds of pressing a bar to be effective.
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
94
Primary and Secondary Reinforcers
267
In psychology, reinforcement is a key concept in behavior modification. B.F. Skinner demonstrated this with his experiments involving rats in what is known as a Skinner box. The rats learned to press a lever to receive food, a primary reinforcer that fulfilled their innate need for nourishment.
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
267


