强大的无奖励的演员-关键的合作多代理强化学习学习
IEEE transactions on neural networks and learning systems
|August 15, 2023
概括
本研究介绍了使用私人奖励的多代理强化学习 (MARL) 的方法,解决了拜占庭式攻击. 一个新的算法,R2F-DDPG,在合作环境中增强了对恶意代理的稳定性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 控制理论 控制理论
背景情况:
- 合作的多代理强化学习 (MARL) 面临着各种各样的挑战,私人奖励功能.
- 集中式培训与分散式执行 (CTDE) 是一个常见的范式.
- 恶意代理人发送虚假信息的拜占庭式攻击威胁到系统完整性.
研究的目的:
- 为CTDE开发一个保护隐私的MARL算法,并提供多种奖励.
- 创建一个强大的MARL算法,能够抵御拜占庭的攻击.
- 引入新的拜占庭式攻击和强大的聚合规则,专门用于强化学习.
主要方法:
- 提出了一个无奖励的深度决定性政策梯度 (RF-DDPG) 算法,转发批评梯度而不是奖励隐私.
- 开发了一个强大的扩展,R2F-DDPG,采用强大的聚合规则来对抗拜占庭袭击.
- 引入了投影增强的强大聚合规则,以解决新型RL特定的拜占庭攻击.
主要成果:
- 在没有拜占庭威胁的情况下,RF-DDPG成功实现了代理人的合作培训.
- R2F-DDPG在复杂的拜占庭袭击中表现出了显著的稳健性.
- 拟议的强有力的聚合规则有效地减轻了RL特定攻击的影响.
结论:
- 该RF-DDPG算法为合作MARL提供了一个保护隐私的解决方案.
- R2F-DDPG为易受拜占庭敌人的MARL系统提供了一个强大的框架.
- 这项工作促进了多代理强化学习系统的安全性和隐私.
相关概念视频
Reinforcement
277
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
277
Reinforcement Schedules
204
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
204
Observational Learning
210
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
210
Avoidance Learning and Learned Helplessness
1.8K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.8K
Associative Learning
444
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
444
Lazarus's Cognitive Appraisal Theory
287
Cognitive psychologist Richard Lazarus proposed the cognitive-mediational theory of emotions, which emphasizes how individuals' assessments of stressors significantly affect their experience of stress. According to Lazarus, the stress response is determined by a two-step appraisal process: primary appraisal and secondary appraisal. These cognitive appraisals help individuals evaluate the potential impact of a stressor and determine the adequacy of their coping resources.
Primary Appraisal:...
Primary Appraisal:...
287


