通过与状态继电器玩等级游戏来增强强化学习
IEEE transactions on neural networks and learning systems
|April 22, 2024
概括
本研究介绍了基于玩游戏与状态继电器 (HGR) 的等级强化学习,以改善深度强化学习 (DRL) 样本利用和探索. 在复杂的运动规划任务中,HGR提高了训练效率.
科学领域:
- 机器人和人工智能 机器人和人工智能
- 机器学习 机器学习
- 运动规划 运动规划
背景情况:
- 深度强化学习 (DRL) 在运动规划中得到广泛应用.
- 当前的DRL方法在任务完成后重置代理状态,导致样本利用率低,环境探索有限.
- 在DRL的初始培训阶段,学习能力较弱,影响复杂任务的效率.
研究的目的:
- 提出一种新的等级增强学习 (HRL) 框架,称为基于玩游戏与状态继电器 (HGR) 的等级学习.
- 提高样本利用率,扩大环境探索,提高复杂运动规划任务的培训效率.
主要方法:
- 引入了一个辅助惩罚来调节任务难度.
- 开发了一个状态中继机制,以利用中间代理状态并扩大低层政策探索.
- 实施了HGR框架,用于复杂环境中的培训代理人.
主要成果:
- 状态中继机制有效地利用中间状态,扩大环境探索.
- 拟议的HGR算法提高了样本利用率,并减轻了稀疏奖励问题.
- 在MazeBase和MuJoCo平台上的模拟测试显示了显著的性能提升.
结论:
- HGR框架有效地解决了动作规划当前DRL的局限性.
- 状态中继机制和辅助惩罚有助于提高采样效率和勘探.
- 对于更广泛的强化学习 (RL) 领域来说,HGR提供了显著的好处.
相关概念视频
Reinforcement
202
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
202
Reinforcement Schedules
144
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
144
Observational Learning
168
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
168
Primary and Secondary Reinforcers
247
In psychology, reinforcement is a key concept in behavior modification. B.F. Skinner demonstrated this with his experiments involving rats in what is known as a Skinner box. The rats learned to press a lever to receive food, a primary reinforcer that fulfilled their innate need for nourishment.
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
Effective reinforcers for humans vary depending on the individual and the context. Primary reinforcers, such as food, water, sleep, shelter, and pleasure, have inherent value and satisfy basic biological...
247
Associative Learning
350
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
350
Avoidance Learning and Learned Helplessness
1.7K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.7K


