通过基于模型的推广,在层次加强学习中进行指导合作
IEEE transactions on neural networks and learning systems
|August 12, 2024
概括
本研究介绍了通过基于模型的推广 (GCMR) 进行指导合作,这是一个针对目标条件下层次增强学习 (HRL) 的新框架. 通过改善信息同步和使用基于模型的推出,GCMR增强了复杂任务的层次间合作和政策改进.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 目标条件下等级强化学习 (HRL) 使用时间抽象来探索复杂的,长时间的任务.
- 有效的层次间沟通和协调对于在等级系统中稳定改进政策至关重要.
- 现有的HRL算法经常忽视层次间的合作,主要关注次目标发现.
研究的目的:
- 通过基于模型的推广 (GCMR) 提出一个新的目标条件的HRL框架,以加强跨层次信息同步和合作.
- 在复杂的强化学习任务中提高样本效率和政策稳定性.
- 在HRL中弥合子目标发现和层次间合作之间的差距.
主要方法:
- 使用基于模型的推出来提高样本效率,减轻政策之外的校正中的状态过渡错误.
- 限制较低水平的Q函数梯度与模型推断的上界,以稳定勘探.
- 实施基于一步推广的规划,高级别的批评者指导低级别的全球信息传输政策.
主要成果:
- 当GCMR框架与邻近性约束和地标导向规划 (ACLG) 集成时,显示出更稳定和更强大的政策改进.
- GCMR显著超过了各种基线算法和以前的最先进的 (SOTA) 方法.
- 实验结果验证了GCMR组件在促进跨层次合作方面的有效性.
结论:
- 通过改进信息同步和利用基于模型的推出,GCMR有效地加强了目标条件的HRL的级别间合作.
- 拟议的框架导致更稳定和更强大的政策改进,超过现有的SOTA算法.
- GCMR为推进复杂的强化学习任务的探索和性能提供了一个有希望的方向.
相关概念视频
Observational Learning
158
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
158
Role of Shaping in Operant Conditioning
284
Shaping is a technique used in operant conditioning to train complex behaviors by rewarding successive approximations toward the target behavior. This method is necessary because organisms are unlikely to perform complex behaviors spontaneously. Instead, shaping breaks down the desired behavior into small, manageable steps.
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
284
Modeling in Therapy
61
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
61
Steps in the Modeling Process
194
Albert Bandura's theory of observational learning identifies four critical processes: attention, retention, motor reproduction, and reinforcement or motivation.
Attention is the first necessary component for observational learning. It involves focusing on what the model is doing and saying. For example, if you decide to take a drawing class to enhance your skills, you need to pay close attention to the instructor's words and hand movements. The characteristics of the model significantly...
Attention is the first necessary component for observational learning. It involves focusing on what the model is doing and saying. For example, if you decide to take a drawing class to enhance your skills, you need to pay close attention to the instructor's words and hand movements. The characteristics of the model significantly...
194
Reinforcement Schedules
139
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
139
Instinctive Drift
200
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
200


