在多代理强化学习中,协调作为推断
Zhiyuan Li1, Lijun Wu1, Kaile Su2
1School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China.
概括
我们介绍了深度运动系统 (DMS),这是一种基于推断的多代理强化学习 (MARL) 的方法. DMS使代理人能够有效地协调,只使用本地信息,克服集中培训,分散执行和独立学习的局限性.
科学领域:
- 人工智能的人工智能
- 强化学习是一种强化学习.
- 多代理系统 多代理系统
背景情况:
- 集中式培训分散式执行 (CTDE) 提升了MARL,但受到了集中式-分散式不匹配 (CDM) 的困扰.
- 完全分散的独立学习 (IL) 模仿了自然的合作,但缺乏代理人意识和协调机制.
研究的目的:
- 提出一个基于推断的协调MARL方法,深度运动系统 (DMS).
- 在独立学习 (IL) 范式中解决代理人意识和协调挑战.
主要方法:
- DMS利用个人意图推断,允许代理人将其他代理人与他们的环境区分开来.
- 用因果推断来加强协调,通过推理对彼此行为的代理效应来加强协调.
主要成果:
- 在多代理MuJoCo和StarCraftII任务中对DMS进行了评估.
- 与独立学习 (IL) 算法相比,提出的方法显示出更高的性能.
- 在不依赖CTDE范式的情况下,DMS成功地学习了协调行为,超过了IPPO和HAPPO等基线.
结论:
- DMS提供了一种有效的MARL协调方法,特别是在分散的环境中.
- 该方法克服了CTDE的局限性,并通过结合意图和因果推理来增强IL.
- 在没有集中培训信息的情况下,DMS在多代理系统中促进了强有力的协调.
相关概念视频
Collisions in Multiple Dimensions: Problem Solving
4.2K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
4.2K
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
Reinforcement Schedules
148
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
148
Associative Learning
375
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
375
Coordination Number and Geometry
15.8K
For transition metal complexes, the coordination number determines the geometry around the central metal ion. Table 1 compares coordination numbers to molecular geometry. The most common structures of the complexes in coordination compounds are octahedral, tetrahedral, and square planar.
15.8K
Collisions in Multiple Dimensions: Introduction
5.4K
It is far more common for collisions to occur in two dimensions; that is, the initial velocity vectors are neither parallel nor antiparallel to each other. Let's see what complications arise from this. The first idea is that momentum is a vector. Like all vectors, it can be expressed as a sum of perpendicular components (usually, though not always, an x-component and a y-component, and a z-component if necessary). Thus, when the statement of conservation of momentum is written for a...
5.4K


