人类战略在强化学习中的适应类似于政策梯度上升
Hua-Dong Xiong1, Li Ji-An2, Robert C Wilson1
1School of Psychology, Georgia Institute of Technology.
bioRxiv : the preprint server for biology
|August 6, 2025
概括
人类随着时间的推移而调整他们的学习策略,类似于基于梯度的优化. 一个新的框架,DynamicRL,量化了这些学习策略的变化,显示了改善的奖励获取和弥合生物和人工智能概念.
科学领域:
- 认知科学 认知科学
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
背景情况:
- 调整学习策略对于智能至关重要,但在生物制剂中缺乏定量框架.
- 现有的计算模型经常假定固定的策略或使用任务优化的网络,无法通过经验来解释策略的改进.
研究的目的:
- 开发一个定量框架来描述生物代理如何调整他们的学习策略.
- 调查人类战略适应是否类似于基于梯度的优化原则.
主要方法:
- 推出了DynamicRL,一个神经网络框架来跟踪参与者的不断变化的学习参数 (学习速度,决策温度).
- 在四个不同的强盗任务中评估了DynamicRL.
主要成果:
- 动态RL的表现优于固定参数的传统强化学习模型.
- 人类学习策略的适应显示了系统增加预期回报的轨迹.
- 战略参数更新与政策梯度上升方向保持一致,并在多个时间范围内运行.
结论:
- 人类动态地调整他们的强化学习策略,与渐变式优化原理保持一致.
- 动态RL框架为研究生物药物的元学习轨迹提供了可通用的方法.
- 这项研究通过经验量化适应性行为优化来弥合生物和人工智能理论的桥梁.
相关概念视频
Natural Selection and Adaptation
386
Natural selection, a fundamental concept in evolutionary biology, is the mechanism by which evolution is driven, favoring organisms that are best adapted to their environments. This process enhances their chances of survival and reproduction. Adaptation, a key outcome of this process, involves genetic modifications that optimize an organism's functionality under specific environmental challenges, such as extreme cold or thinner air at high altitudes.
Beyond physical adaptations,...
Beyond physical adaptations,...
386
Avoidance Learning and Learned Helplessness
1.9K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.9K
Behavior Modification
236
Behavioral approaches have often been criticized for ignoring mental processes and focusing solely on observable behavior. However, these approaches provide an optimistic perspective for individuals seeking to change their behaviors. Rather than concentrating on intrinsic personality traits, behavioral approaches suggest that even longstanding habits can be modified by changing the reward contingencies that maintain them.
A real-world application of operant conditioning principles is applied...
A real-world application of operant conditioning principles is applied...
236
Observational Learning
314
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
314
Law of Effect
1.6K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
1.6K
Evolutionary Psychology
424
Evolutionary psychology explores the origins of human behavior and mental processes by framing them within the context of natural selection, a theory famously propounded by Charles Darwin. This field asserts that many behaviors common across human societies — ranging from instinctive fear reactions to complex social interactions — arose as evolutionary adaptations. These adaptations enhanced the survival and reproductive success of our ancestors, thereby becoming embedded in the...
424


