多巴胺过渡物编码奖励预测错误,独立于学习率
Andrew Mah1, Carla E M Golden1, Christine M Constantinople1
1Center for Neural Science, New York University.
bioRxiv : the preprint server for biology
|April 25, 2024
概括
核心核中的多巴胺编码奖励预测错误 (RPEs),但不是强化学习期间的学习率. 这表明多巴胺独立机制在不稳定的环境中动态调整学习速度.
科学领域:
- 神经科学是一个神经科学.
- 计算精神病学是一种计算精神病学.
- 强化学习是一种强化学习.
背景情况:
- 理论上,多巴胺编码奖励预测错误 (RPEs),乘以学习速率来更新强化学习中的值.
- 纹状体中的突触可塑性被认为是这些更新的媒介,这意味着多巴胺释放反映了学习速率和RPE的产物.
研究的目的:
- 在强化学习过程中调查多巴胺在核核 (NAcc) 中对学习速度的编码.
- 确定多巴胺释放是否反映了RPE和动态学习率,特别是在波动的环境中.
主要方法:
- 鼠被训练在一个任务与半可观察状态和不同的奖励.
- 用行为分析和计算建模来分析学习率和信念更新的试验对试验调整.
- 在NAcc中释放的多巴胺在任务执行期间被测量.
主要成果:
- 在状态过渡之后,学习率更高,并与对隐藏状态的信念变化进行缩放,与贝叶斯学习保持一致.
- 在NAcc中释放的多巴胺编码了RPE,但独立于计算的学习率.
- 根据RPE调整试验启动的老鼠,证明了自适应性学习.
结论:
- 在NAcc中的多巴胺编码RPEs,与现有理论相一致.
- 这些发现表明,多巴胺独立的机制负责实例化动态学习速率.
- 这挑战了多巴胺释放直接反映学习速率和RPE的产物的概念.
相关概念视频
Instinctive Drift
208
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
208
Timing and Consequences on Behavior
90
In operant conditioning, the timing of reinforcement is crucial. For animals like rats and cats, immediate reinforcement (within a few seconds) is much more effective than delayed reinforcement. For example, a food reward for a rat needs to follow within 30 seconds of pressing a bar to be effective.
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
90


