屏蔽计划指导数据高效和安全的增强学习学习
IEEE transactions on neural networks and learning systems
|February 6, 2024
概括
本研究介绍了屏蔽规划引导的政策优化 (SPPO),一种新的安全强化学习 (RL) 方法. SPPO确保了训练期间的安全,并提高了机器人系统的勘探效率.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 人工智能的人工智能
- 控制理论 控制理论
背景情况:
- 安全增强学习 (RL) 对通用机器人系统至关重要.
- 现有的方法主要关注培训后的安全,忽视培训中的安全和探索效率.
研究的目的:
- 开发基于模型的安全RL方法,确保训练期间的安全并改善勘探.
- 为增强机器人学习引入屏蔽规划引导的政策优化 (SPPO).
主要方法:
- SPPO将路径规划和屏蔽机制集成到政策优化中.
- 使用模型预测路径集成 (MPPI) 进行屏蔽勘探和数据收集.
- 采用基于优势的屏蔽规则和以任务为导向的参数优化 (TOPO) 来改进政策.
主要成果:
- 与基线方法相比,SPPO表现出优越的政策绩效.
- 在培训阶段实现更高的学习效率和安全性能.
- 为培训安全,部署安全和政策执行提供明确的理论保证.
结论:
- SPPO有效地解决了在机器人技术的RL中培训安全和探索效率的挑战.
- 该方法为开发更安全,更高效的通用机器人系统提供了强大的框架.
- 理论上的保证和实验结果验证了SPPO.的有效性.
相关概念视频
Role of Shaping in Operant Conditioning
322
Shaping is a technique used in operant conditioning to train complex behaviors by rewarding successive approximations toward the target behavior. This method is necessary because organisms are unlikely to perform complex behaviors spontaneously. Instead, shaping breaks down the desired behavior into small, manageable steps.
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
322
Reinforcement Schedules
148
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
148
Reinforcement
209
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
209
Law of Effect
1.4K
B.F. Skinner, a prominent figure in behavioral psychology, introduced operant conditioning by emphasizing the role of consequences in shaping behavior. This theory builds upon the law of effect proposed by Edward Thorndike, which posits that behaviors followed by satisfying outcomes are likely to be repeated. In contrast, those followed by unsatisfying outcomes are less likely to recur.
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
Edward Thorndike's foundational work involved studying learning in animals, particularly using puzzle...
1.4K
Generalization, Discrimination, and Extinction
559
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
559
Elaborative Rehearsals
86
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
86


