在无模型的受约束强化学习中学习零约束违规安全政策.
IEEE transactions on neural networks and learning systems
|January 17, 2024
概括
本研究介绍了一种用于安全强化学习 (RL) 的新算法,该算法可以学习政策而不会违反任何约束. 通过使用以安全为导向的能源功能,它在复杂的环境中实现零违规性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 无模型的强化学习 (RL) 经常与安全限制作斗争,因为传统方法需要经历危险的情况来学习避免.
- 在RL中现有的后面惩罚方法不能保证零约束违规,即使在趋同后,也限制了它们在安全关键领域的适用性.
研究的目的:
- 开发一种无模型的强化学习方法,能够学习零约束违规安全政策.
- 提出一种新的算法,即安全设置行动者-关键 (SSAC),它利用能量功能来提高安全性.
主要方法:
- 使用数据驱动的方法来学习以安全为导向的能量功能,消除了对已知的环境动态的需求.
- 制定一个受约束的强化学习问题,以利用基于拉格朗的方法推导和优化零违规政策.
主要成果:
- 拟议的SSAC算法成功地学习了在复杂的模拟环境中实现零约束违规的策略.
- 实验验证,包括使用自主车辆控制器进行硬件循环测试,证明了该方法的有效性.
- 学习的政策表现出与基于模型的基线可比的性能,同时确保绝对的安全性.
结论:
- SSAC算法有效地解决了在没有模型的RL中学习零约束违规安全策略的挑战.
- 能源功能和受限制的RL的整合为自主系统中安全决策提供了强大的框架.
- 这项工作为在现实场景中更安全,更可靠的RL应用铺平了道路.
相关概念视频
01:26Constraints and Statical Determinacy
608
In structural engineering, the equilibrium of a system is not only determined by its equations of equilibrium but also with the help of constraints. Constraints refer to restrictions on the motion of a system. The proper combinations of constraints can minimize the total number of constraints needed to maintain a system in mechanical equilibrium. When this happens, the system is said to be statically determinate. For such systems, the unknown reaction supports can be estimated using equilibrium...
608
01:24Reinforcement Schedules
148
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
148
01:16Statically Indeterminate Problem Solving
381
Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
381
01:13Stability of Equilibrium Configuration: Problem Solving
606
The stability of equilibrium configurations is an important concept in physics, engineering, and other related fields. In simple terms, it refers to the tendency of an object or system to return to its equilibrium position after being disturbed. The stability of an equilibrium configuration can be analyzed by considering the potential energy function of the system and examining its behavior near the equilibrium point.
Problem-solving in the context of the stability of equilibrium configuration...
Problem-solving in the context of the stability of equilibrium configuration...
606
01:23Reinforcement
211
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
211
01:27Punishment
192
Negative reinforcement and punishment are often confused but serve distinct functions in behavior modification. Reinforcement, whether positive or negative, increases the likelihood of a desired behavior, while punishment decreases it.
Punishment can be positive or negative. Positive punishment involves adding an undesirable stimulus, such as scolding, to decrease a behavior. Negative punishment involves removing a desirable stimulus, such as taking away a favorite toy, to decrease behavior....
Punishment can be positive or negative. Positive punishment involves adding an undesirable stimulus, such as scolding, to decrease a behavior. Negative punishment involves removing a desirable stimulus, such as taking away a favorite toy, to decrease behavior....
192