通过认知一致性重新思考通过强化学习的探索-开发权衡
1Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education and the School of Computer and Information Technology, Shanxi University, Taiyuan 030006, Shanxi, China.
概括
本研究介绍了深度强化学习 (RL) 的认知一致性 (CoCo) 框架. 通过悲观的探索,CoCo通过保持认知一致性来提高样本的效率和性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度强化学习 (deep reinforcement learning) 是一种深度强化学习的方法.
背景情况:
- 勘探-开采困境是深度强化学习 (RL) 中的一个核心挑战.
- 现有的勘探方法往往缺乏采样效率.
- 人类保持认知的一致性,以便有效地做出决策.
研究的目的:
- 提出一种新的框架,即认知一致性 (CoCo),用于深度强化学习.
- 通过重新思考勘探-开发权衡,提高RL中的样本效率和性能.
- 为了利用认知一致性原则来实现更智能的代理行为.
主要方法:
- 开发了认知一致性 (CoCo) 框架.
- 为了最佳的政策认知,利用了自我模仿的分布校正.
- 实施悲观的探索,以不一致性最小化目标,灵感来自标签分发学习.
主要成果:
- 在标准的非政策RL任务上验证了CoCo框架.
- 保持认知的一致性明显提高了样本效率.
- 在经过测试的RL任务中观察到提高了性能.
结论:
- 认知一致性提供了一种有希望的方法来增强RL.
- CoCo框架有效地平衡了勘探和开采.
- 这种方法提高了RL代理人的学习速度和质量.
相关概念视频
Cognitive Learning
131
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
131
Cognitive Dissonance
32.4K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
32.4K
The Anchoring-and-Adjustment Heuristic
7.2K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.2K
Instinctive Drift
173
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
173
Generalization, Discrimination, and Extinction
383
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
383
Decision Making: P-value Method
5.2K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.2K


