基于图形的多代理强化学习与进化人口合作合作
Kexing Peng1, Hanwen Qi1, Tinghuai Ma2
1School of Computer Science, Nanjing University of Information Science & Technology, Nanjing, 210044, China.
概括
本研究介绍了GDE,这是一个新的多代理强化学习 (MARL) 框架,可以提高复杂任务的协调. GDE 结合了基于图的值分解与分阶段的进化策略优化,以提高代理性能.
科学领域:
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
- 计算机科学 计算机科学
背景情况:
- 由于有限的代理观察和动态相互作用,现有的多代理强化学习 (MARL) 方法在扩展到复杂的协调任务方面面临挑战.
- 由于任务的复杂性和政策空间的增加,趋同到最佳政策是很困难的,这会影响稳定的政策评估.
研究的目的:
- 提出GDE,一个MARL框架,旨在克服合作多代理系统中的可扩展性和融合问题.
- 在动态环境中加强代理协调和信息传播,而不需要国家共识.
主要方法:
- GDE集成基于图的值分解与分阶段的进化政策优化.
- 进化算法 (EA) 用于无梯度的随机搜索,以改善政策探索和趋同.
- 图形神经网络 (GNN) 用于扩展代理受体场并促进信息传播,利用顺序不变来与动态数据稳定融合.
主要成果:
- 在复杂的协调任务中,GDE表现出卓越的性能,包括StarCraft II微管理,MAMuJoCo机器人合作和SUMO自动驾驶.
- 该框架通过多代理人团队组建和GNN有效地捕捉了复杂的协调动态.
- 实验结果验证了GDE框架内每个模块的有效性和必要性.
结论:
- 在MARL中,GDE提供了一个强大的解决方案,以加强协调和政策融合.
- 基于图形的分解和进化优化的建议组合对复杂的多代理系统有效.
- 该框架的模块化设计和适应性使其适用于各种现实应用.
相关概念视频
01:23Reinforcement
786
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
786
00:48Optimal Foraging
13.4K
How animals obtain and eat their food is called foraging behavior. Foraging can include searching for plants and hunting for prey and depends on the species and environment.
13.4K
01:12Observational Learning
791
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
791
01:06Collisions in Multiple Dimensions: Problem Solving
5.2K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
5.2K
02:54Evolutionary Relationships through Genome Comparisons
6.8K
Genome comparison is one of the excellent ways to interpret the evolutionary relationships between organisms. The basic principle of genome comparison is that if two species share a common feature, it is likely encoded by the DNA sequence conserved between both species. The advent of genome sequencing technologies in the late 20th century enabled scientists to understand the concept of conservation of domains between species and helped them to deduce evolutionary relationships across diverse...
6.8K
01:22Multi-input and Multi-variable systems
373
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
373