通过自我蒸消除在线强化学习中的优先偏见.
概括
在线强化学习中的优先偏见导致过度拟合. 自蒸增强学习 (SDRL) 将知识转移到新政策中,减轻偏见并改善概括,以获得更好的代理性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 深度强化学习在训练的早期容易过度适应.
- 这种过度匹配,称为优先偏差,导致低于最佳的代理决策和探索.
- 优势偏见阻碍代理人学习未来国家的通用政策.
研究的目的:
- 系统地调查在线强化学习中优先偏见的原因和特征.
- 开发一个新的框架,自蒸强化学习 (SDRL),以克服优先偏见.
- 增强政策的通用化,加快学习的强化学习代理.
主要方法:
- 开发了一个基于知识蒸的自我蒸强化学习 (SDRL) 框架.
- 代理人将学到的知识转移到一个新的,随机初始化的政策中,定期间隔.
- 整合了L2规范化和自我模仿机制,以防止过度匹配和加速学习.
主要成果:
- 在DMC和Atari 100k上的实验结果证明了SDRL在消除优先偏差方面的有效性.
- 拟议的方法可以产生更普遍的政策.
- 通过知识蒸提炼的政策可以更快地改善代理商的成绩.
结论:
- 自蒸增强学习有效地解决了在线增强学习中的优先偏见.
- SDRL促进了强有力的和普遍的政策的发展.
- 该框架提高了复杂环境中的学习效率和代理性能.
相关概念视频
Self-Discrepancy Theory
18.3K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.3K
Instinctive Drift
208
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
208
Behaviorism
2.3K
The field of behaviorism was pioneered by figures such as Ivan Pavlov, John B. Watson, and B.F. Skinner fundamentally shifted the focus of psychology to the observable and controllable aspects of human and animal behavior. This shift marked a critical evolution in the discipline, emphasizing scientific rigor and experimental methodology.
The core premise of behaviorism is its focus on observable behavior rather than internal thoughts or feelings. This approach argues that true scientific...
The core premise of behaviorism is its focus on observable behavior rather than internal thoughts or feelings. This approach argues that true scientific...
2.3K
Stereotype Threat and Self-fulfilling Prophecies
37.6K
When we hold a stereotype about a person, we have expectations that he or she will fulfill that stereotype. A self-fulfilling prophecy is an expectation held by a person that alters his or her behavior in a way that tends to make it true. When we hold stereotypes about a person, we tend to treat the person according to our expectations. This treatment can influence the person to act according to our stereotypic expectations, thus confirming our stereotypic beliefs. Research by Rosenthal and...
37.6K
Timing and Consequences on Behavior
90
In operant conditioning, the timing of reinforcement is crucial. For animals like rats and cats, immediate reinforcement (within a few seconds) is much more effective than delayed reinforcement. For example, a food reward for a rat needs to follow within 30 seconds of pressing a bar to be effective.
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
Humans, however, can respond to delayed reinforcers. We often make decisions between immediate small rewards and delayed larger rewards. This ability to delay gratification is a significant...
90
Generalization, Discrimination, and Extinction
532
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
532


