通过案例学习和好奇心实现协作多代理的双驱优化
Ruizhu Chen1, Rong Fei2, Junhuai Li2
1School of Computer Science and Engineering, Xi'an University of Technology, Xi'an, 710048, shaanxi, China.
概括
本研究引入了一种用于多代理深度强化学习 (MADRL) 的新方法,该方法可以改善勘探和开发. 案例增强的随机网络蒸探索 (CERE-CTDE) 范式提高了复杂场景中的学习效率和稳定性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 机器人技术 机器人技术 机器人技术
背景情况:
- 多代理深度强化学习 (MADRL) 在培训期间与勘探-开发权衡作斗争.
- 现有的勘探方法往往缺乏目标方向,导致数据收集效率低下和不稳定的趋同.
- 稀少的奖励和需要协作行为的需要在MADRL中提出了重大挑战.
研究的目的:
- 提出一个新的范式,CASE-Enhanced Random Network Distillation Exploration for Centralized Training and Decentralized Execution (CERE-CTDE),以解决MADRL培训方面的挑战. 为了解决MADRL培训方面的挑战,我们提出了一个新的范式,CASE-Enhanced Random Network Distillation Exploration for Centralized Training and Decentralized Execution (CERE-CTDE),用于解决MADRL培训方面的挑战.
- 通过随机网络蒸 (RND) 和基于案例的推理 (CBR) 的协同集成,增强MADRL的勘探开发平衡.
- 提高学习效率,政策趋同的稳定性,以及在MADRL中逃避局部最佳的能力.
主要方法:
- 随机网络蒸 (RND) 的整合,以获得内在动机和增强探索.
- 纳入基于案例的推理 (CBR) 以利用历史数据实现目标导向的利用.
- 在MADRL的集中培训和分散执行 (CTDE) 框架内应用CERE范式.
主要成果:
- 在复杂的StarCraft多代理挑战 (SMAC) 场景中,赢得率在统计学上显著提高了17.97%.
- 通过内在动机和CBR指导的行动抽样,有效地加强了政策探索-利用,并减轻了稀缺奖励问题.
- 在保持高学习效率的同时,展示了在逃避局部最佳的卓越能力.
结论:
- CERE-CTDE范式为改善MADRL性能提供了强大的解决方案,特别是在奖励稀疏和相互作用复杂的环境中.
- RND和CBR的双重机制有效地平衡了勘探和开发,从而导致更稳定和更有效的学习.
- 该框架在SMAC的不同难度级别中的一致性能验证了其稳定性和实际适用性.
相关概念视频
Purposive Learning
444
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
444
Observational Learning
838
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
838
Collisions in Multiple Dimensions: Problem Solving
5.3K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
5.3K
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Cognitive Learning
1.0K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.0K
Machines: Problem Solving II
647
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. Consider a lifting tong carrying a 100 kg load. It comprises movable sections DAF and CBG linked together with member AB.
647
