轻量级的变压器图像特征提取网络
Wenfeng Zheng1, Siyu Lu1, Youshuai Yang1
1School of Automation, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.
PeerJ. Computer science
|December 13, 2024
概括
图像特征提取的变压器模型面临由于二次复杂性的计算挑战. 这项研究引入了一种高效的注意力机制,可将计算量降低高达70%,以更快地处理高分辨率图像.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 变压器模型越来越多地用于图像特征提取.
- 变压器中自我注意的二次复杂性限制了高分辨率图像处理,并增加了计算成本.
- 现有的方法难以满足大规模图像数据的计算需求.
研究的目的:
- 开发高效的变压器模型用于图像特征提取.
- 为了解决自我注意力机制中的二次复杂性问题.
- 为了实现高分辨率图像的处理,降低了计算成本.
主要方法:
- 将自我注意机制的二次复杂度降低到线性复杂度.
- 引入了一个无参数的轻量级修剪方法来过不重要的令牌.
- 结合了这些方法来创建一个高效的注意力机制.
主要成果:
- 结合的方法使原始变压器模型的计算减少了30%-50%.
- 高效的注意力机制实现了60% - 70%的计算减少.
- 证明了对高分辨率图像的改进处理能力.
结论:
- 提出的高效注意力机制显著减少了变压器模型中的计算负载.
- 这种方法提高了使用变压器进行高分辨率图像分析的可行性.
- 这些方法为计算上昂贵的图像特征提取任务提供了实用解决方案.
相关概念视频
The Ideal Transformer
354
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
354
Types Of Transformers
946
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
946
Energy Losses in Transformers
828
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
828
Transformers in Distribution System
98
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
98
Reducing Line Loss
144
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
144
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K


