安全的适应性政策转移强化学习,用于分布式多代理控制
IEEE transactions on neural networks and learning systems
|November 2, 2023
概括
本研究介绍了一种安全的自适应性政策转移强化学习 (RL) 方法. 它使后续代理从先驱代理学习,改善合作控制和安全.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 多代理强化学习 (RL) 培训是具有挑战性的,因为代理干扰和安全限制.
- 现有的方法在合作的多代理系统中难以有效地转移知识和适应性学习.
研究的目的:
- 为多代理合作控制提出安全的适应性政策转移RL方法.
- 通过知识转移,提高多代理系统的学习效率和安全性.
主要方法:
- 引入了一种先驱和追随者非政策政策转移学习 (PFOPT) 方法.
- 使政策代表和样本经验从先驱转移到追随者代理.
- 利用瓦瑟斯坦距离,在先前的经验和探索之间适应调整学习权重.
主要成果:
- 训练有素的分布式代理成功完成了协作任务,最大限度地提高了奖励,同时最大限度地减少了约束违规行为.
- 与基线方法相比,在学习速度和成功率方面表现令人满意.
- 通过PFOPT方法,有效地转移知识,并根据政策分配差异调整学习.
结论:
- 提出的安全适应性政策转移RL方法显著改善了多代理合作控制.
- 对于复杂的多代理培训场景,PFOPT提供了一种高效和安全的解决方案.
- 这种方法为分散式系统中的知识共享和适应性学习提供了强大的框架.
相关概念视频
Distributed Loads: Problem Solving
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
State Space to Transfer Function
The conversion of state-space representation to a transfer function is a fundamental process in system analysis. It provides a method for transitioning from a time-domain description to a frequency-domain representation, which is crucial for simplifying the analysis and design of control systems.
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
Multi-input and Multi-variable systems
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
Distribution Reliability and Automation
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
Reinforcement
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Observational Learning
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...


