Int-HRL:朝着基于意图的层次强化学习的学习方向
Anna Penzkofer1, Simon Schaefer2, Florian Strohm1
1Institute for Visualisation and Interactive Systems, University of Stuttgart, Pfaffenwaldring 5A, 70569 Stuttgart, Germany.
Neural computing & applications
|August 4, 2025
概括
本研究介绍了Int-HRL,一种新的层次强化学习 (RL) 方法. 通过使用人类眼睛的目光来预测意图,它会自动创建子目标,提高挑战性RL任务的样本效率.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 深度强化学习 (RL) 代理人擅长执行任务,但需要大量的数据进行培训.
- 层次式RL (HRL) 使用结构信息提高了样本效率,但依赖于人类注释的子目标.
- 发现有效的子目标是HRL在复杂,长期任务中的一个主要挑战.
研究的目的:
- 开发一种新的HRL方法,减少对人类注释子目标的需求.
- 为了利用人类的意图预测从眼睛的目光自动化子目标的生成.
- 在具有挑战性的RL环境中提高样本效率,如蒙特祖马的复仇.
主要方法:
- 从目光数据预测人类玩家的意图.
- 根据预测的意图,开发一个自动的次目标提取管道.
- 实施基于意图的层次增强学习 (Int-HRL).
主要成果:
- 人类的意图可以从长远的视野,稀疏的奖励任务的眼神中得到强有力的预测.
- 拟议的自动次目标提取管道有效地取代了手动注释.
- 与以前的HRL方法相比,Int-HRL显示了显著提高的样本效率.
结论:
- 基于眼睛的目光的意图预测提供了一个可行的替代方案,可以在HRL中手动注释子目标.
- Int-HRL显著提高了样本效率,使复杂的RL任务更容易处理.
- 这种方法为更自主和更有效的学习代理铺平了道路.
相关概念视频
Reinforcement Schedules
242
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
242
Reinforcement
342
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
342
Purposive Learning
207
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
207
Law of Effect
1.6K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
1.6K
Associative Learning
576
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
576
Operant Conditioning Intervention
114
Operant conditioning serves as a foundational principle in therapeutic interventions aimed at modifying maladaptive behaviors. Central to this approach is the notion that behaviors, both adaptive and maladaptive, are learned through reinforcement. By analyzing the environmental factors that reinforce problematic behaviors, clinicians can design interventions to weaken these reinforcements and replace maladaptive behaviors with healthier alternatives.
In operant conditioning, behaviors that are...
In operant conditioning, behaviors that are...
114


