相关实验视频
Updated: Jul 1, 2025

09:45
New Variations for Strategy Set-shifting in the Rat
Published on: January 23, 2017
8.2K
动态增强学习揭示了奖励学习期间战略的时间依赖转变
bioRxiv : the preprint server for biology
|March 11, 2024
概括
这项研究引入了一种新的模型,以了解大脑系统在学习过程中如何改变策略. 该模型揭示了基于模型和无模型学习过程的动态变化,影响行为和神经活动.
科学领域:
- 神经科学是一个神经科学.
- 计算神经科学是一种神经科学.
- 强化学习是一种强化学习.
背景情况:
- 行为通常由竞争性大脑系统来解释,无模型 (MF) 和基于模型 (MB) 的强化学习代理被假设代表自动性和审议.
- 静态代理混合 (MoA) 模型捕捉了这些差异,但无法解释动态战略转变.
研究的目的:
- 为了开发一种新的计算框架,混合代理隐藏的马尔科夫模型 (MoA-HMM) 能够捕捉动态策略的行为变化.
- 调查代理贡献的时间动态及其与奖励导向任务期间行为和神经变化的关系.
主要方法:
- 开发了MoA-HMM,可以同时从多个代理推断动作值,并建模潜在隐藏状态的时间动态.
- 将MoA-HMM应用于执行多步骤,奖励导向任务的老鼠的行为数据.
主要成果:
- MoA-HMM揭示了老鼠的会话内策略的进展,从初始的MB探索转向MB利用,最终减少参与.
- 推断的隐藏状态预测了任务期间响应时间和轨道前皮层 (OFC) 神经编码的显著变化.
结论:
- MOA-HMM提供了一个动态的框架,以了解竞争性学习系统如何相互作用和随着时间的推移而变化.
- 已识别的隐藏状态代表了认知动态的真正转变,影响了行为和神经表征.
相关概念视频
Reinforcement Schedules
145
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
145
Timing and Consequences on Behavior
90
In operant conditioning, the timing of reinforcement is crucial. For animals like rats and cats, immediate reinforcement (within a few seconds) is much more effective than delayed reinforcement. For example, a food reward for a rat needs to follow within 30 seconds of pressing a bar to be effective.
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
90
Instinctive Drift
218
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
218
Reinforcement
206
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
206
Primary and Secondary Reinforcers
253
In psychology, reinforcement is a key concept in behavior modification. B.F. Skinner demonstrated this with his experiments involving rats in what is known as a Skinner box. The rats learned to press a lever to receive food, a primary reinforcer that fulfilled their innate need for nourishment.
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
253
Law of Effect
1.4K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
1.4K

