无监督强化学习的自我引用代理
Andrew Zhao1, Erle Zhu2, Rui Lu1
1Department of Automation, BNRist, Tsinghua University, China.
概括
本研究介绍了自我参考 (SR),这是一个用于无监督强化学习的附加模块. 通过稳定奖励和保持探索性行为,SR提高了性能和样本效率.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 强化学习是一种强化学习.
背景情况:
- 无监督强化学习 (URL) 方法在预训练期间经常与奖励非静止性作斗争.
- 在URL的微调阶段,探索性行为可能会被遗忘.
研究的目的:
- 介绍自我引用 (SR),这是一个新的URL附加模块.
- 地址奖励非静态性和忘记URL中的探索性行为.
主要方法:
- 在预训练期间,SR使用历史引用来稳定内在的奖励.
- 在微调过程中,SR保留了有价值的探索行为.
主要成果:
- 显著提高了现有的URL无模型方法的性能和样本效率.
- 提高了高达17%的内在质量指标 (IQM).
- 在无监督强化学习基准上将优化差距减少31%.
结论:
- SR是一个适用于现有的URL方法的通用和兼容的附加模块.
- SR有效地减轻了奖励的非静止性,并保留了探索性技能.
相关概念视频
Observational Learning
106
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
106
Reinforcement Schedules
119
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
119
Self-Evaluation: Self-Enhancement and Self-Verification
5.1K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.1K
Self-Presentation: Self-Monitoring and Self-Handicapping
38.6K
People can go to great lengths to protect their self-image and present themselves in ways that they want others to see them. Sociologist Erving Goffman presented the idea that a person is like an actor on a stage. Calling his theory dramaturgy, Goffman believed that we use “impression management” to present ourselves to others as we hope to be perceived. Each situation is a new scene, and individuals perform different roles depending on who is present (Goffman, 1959). Think about...
38.6K
Self-Schemas
30.8K
In general, a schema is a mental construct consisting of a cluster or collection of related concepts (Bartlett, 1932). There are many different types of schemata, and they all have one thing in common: schemata are a method of organizing information that allows the brain to work more efficiently. When a schema is activated, the brain makes immediate assumptions about the person or object being observed.
30.8K
Nonconscious Mimicry
4.5K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.5K


