一个混合在线非政策强化学习代理框架,由变压器支持
Enrique Adrian Villarrubia-Martin1, Luis Rodriguez-Benitez1, Luis Jimenez-Linares1
1Department of Technologies and Information Systems, Universidad de Castilla-La Mancha, Paseo de la Universidad 4, 13005 Ciudad Real, Spain.
International journal of neural systems
|October 19, 2023
概括
本研究介绍了一种使用变压器的混合强化学习 (RL) 代理来提高决策效率. 该方法增强了在线代理商的培训,特别是在有限的数据的情况下.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 强化学习 (RL) 使代理人能够通过环境互动来学习最佳政策.
- 传统的RL面临着诸如广泛的数据要求和长期信用分配等挑战.
- 变压器在解决RL的局限性方面表现有前途,特别是在离线环境中.
研究的目的:
- 提出一个框架,增强在线非政策强化学习代理使用变压器.
- 通过自我注意机制解决RL中的数据低效和信用分配问题.
- 提高基于变压器的RL代理人在数据稀缺或未知的环境中的培训效率.
主要方法:
- 一个混合代理,结合了在线非政策RL代理和离线变压器代理 (决策变压器架构).
- 在线和离线代理之间的经验重播缓冲器的连续交换.
- 利用变压器架构内在的自我注意机制.
主要成果:
- 在混合代理的初始代中提高了学习训练效率.
- 加强基于变压器的RL代理人的培训,特别是在数据有限的场景中.
- 混合政策方法在应对RL挑战方面表现出的有效性.
结论:
- 拟议的框架有效地提高了使用变压器的在线非政策RL培训.
- 混合代理架构和经验重复交换提高了学习效率和数据利用.
- 这种方法为RL在具有有限数据或未知的动态的具有挑战性的环境中提供了一个有前途的解决方案.
相关概念视频
Transformers in Distribution System
105
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
105
Types Of Transformers
987
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
987
Transformers with Off-Nominal Turns Ratios
162
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
162
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
The Ideal Transformer
407
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
407
Reinforcement
221
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
221


