对变压器的调查以及使用变压器进行自然语言处理的集体学习
Hongzhi Zhang1, M Omair Shafiq1
1School of Information Technology, Carleton University, Ottawa, ON Canada.
概括
本研究将BERT,XLNet,RoBERTa,GPT2和ALBERT等变压器模型与各种自然语言处理任务进行比较. 集体学习模型在特定任务中表现出比单个分类器更高的性能.
科学领域:
- 自然语言处理自然语言处理.
- 深度学习 (Deep Learning) 是一种深度学习.
- 人工智能的人工智能
背景情况:
- 谷歌于2017年推出的变压器模型彻底改变了自然语言处理 (NLP).
- 像BERT,XLNet,RoBERTa和ALBERT这样的预训练模型在各种NLP任务中取得了显著的成功.
- 现有的研究提供了这些先进的变压器架构的有限的全面比较.
研究的目的:
- 系统地描述和比较一些著名的变压器模型 (BERT,XLNet,RoBERTa,GPT2,ALBERT).
- 评估这些模型在一系列既定的NLP任务上的表现.
- 引入和评估集体学习模型,以提高NLP任务性能.
主要方法:
- 对BERT,XLNet,RoBERTa,GPT2和ALBERT模型进行比较分析.
- 在六个关键NLP任务上应用和评估模型:情绪分析,问题答案,文本生成,文本总结,命名实体识别和主题建模.
- 开发和测试集体学习模型,集成现有变压器架构.
主要成果:
- 性能分析揭示了不同NLP任务中各个变压器模型的不同优缺点.
- 集体学习模型在特定的评估任务上始终表现优于单个变压器模型.
- 这项研究提供了实证证据,证明了在NLP中整体方法的有效性.
结论:
- 变压器模型为广泛的NLP挑战提供了强大的功能.
- 集合方法代表了一个有前途的方向,可以使NLP任务性能超出个别模型的局限性.
- 对专业集体策略的进一步研究可以在复杂的NLP应用中释放更大的潜力.
相关概念视频
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
Types Of Transformers
976
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
976
Transformers in Distribution System
103
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
103
Transformers with Off-Nominal Turns Ratios
154
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
154
The Ideal Transformer
394
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
394
Energy Losses in Transformers
875
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
875


