在视觉变压器下层的频率辅助局部注意力
Xin Zhou1, Zeyu Jiang1, Shihua Zhou1
1School of Mechanical Engineering and Automation, Northeastern University, Wenhua Road, Shen Yang, Liao Ning, P. R. China.
International journal of neural systems
|February 27, 2025
概括
本研究介绍了频率辅助局部注意力变压器 (FALAT),这是一种新的视觉模型,可以有效地捕捉局部特征而无需卷积块. 通过整合频域信息和专用注意力机制,FALAT提高了图像识别任务的性能.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 视觉转换器 (ViT) 在全球特征关系方面表现出色,但在本地特征捕获方面却很难,通常需要卷积神经网络 (CNN).
- 现有的ViT需要卷积辅助来有效地整合本地特征信息,从而限制了它们的独立能力.
- 有需要的变压器架构,可以固有的处理本地信息,而不依赖于传统的卷积块.
研究的目的:
- 提出一种新的混合多尺度模型,即频率辅助局部注意力变压器 (FALAT),能够有效地捕捉局部特征.
- 为了证明变压器模型可以使用专门的初始化和注意力机制来处理本地信息,模仿卷积内核.
- 通过结合频域数据和高效的卷积替换来增强信息多样性和模型性能.
主要方法:
- 引入基于频率辅助窗口的定位自我注意 (FWPSA) 模块,以限制注意范围和捕捉早期的本地内容.
- 利用价值代币的频率域信息来丰富自我注意力计算并增强信息多样性.
- 在空间缩小注意力模块中用深度可分离的卷曲取代传统的下方采样卷曲,以实现高效的远程依赖处理.
主要成果:
- 在ImageNet-1k (IN-1k) 上,FALAT-S实现了83.0%的精度,参数为29.9M,FLOP为5.6G.
- 在使用Mask R-CNN时,在COCO数据集上表现比Next-ViT-S高0.9%AP/0.8%AP.
- 在使用FPN的ADE20k数据集上超过FastViT-SA36的3.1%mIoU.
结论:
- 拟议的FALAT模型表明,变压器可以通过专门的注意力机制和初始化有效地捕捉局部特征,而不需要明确的卷积块.
- FALAT提供了一个有前途的混合架构,可以平衡全球和本地特征提取,从而在各种计算机视觉基准上实现最先进的性能.
- 集成频域信息和高效的卷积技术有助于提高模型的准确性和效率.
相关概念视频
01:21Association Areas of the Cortex
4.9K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
4.9K
01:24Vision
52.9K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.9K
01:20Parallel Processing
143
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
143
01:10Fast Fourier Transform
252
The Fast Fourier Transform (FFT) is a computational algorithm designed to compute the Discrete Fourier Transform (DFT) efficiently. By breaking down the calculations into smaller, manageable sections, the FFT significantly reduces the computational complexity involved. Direct computation of an N-point DFT requires N2 complex multiplications, whereas the FFT algorithm needs only (N/2)log2N multiplications, offering a much faster performance.
The computational efficiency of the FFT becomes...
The computational efficiency of the FFT becomes...
252