SVE-Former:

Xiang-Jun Shen1, Wenxiu Tian1, Yang Yang1

  • 1School of Computer Science and Communication Engineering, Jiangsu University, Jiangsu University Campus, Zhenjiang, 212013, Jiangsu, China.

概括

SVE-Former引入了单向量嵌入,通过捕获令牌依赖来提高变压器效率. 这种方法显著减少了培训时间,并提高了对远程依赖的性能.

相关概念视频

Fast Fourier Transform01:10

Fast Fourier Transform

The Fast Fourier Transform (FFT) is a computational algorithm designed to compute the Discrete Fourier Transform (DFT) efficiently. By breaking down the calculations into smaller, manageable sections, the FFT significantly reduces the computational complexity involved. Direct computation of an N-point DFT requires N2 complex multiplications, whereas the FFT algorithm needs only (N/2)log⁡2N multiplications, offering a much faster performance.
The computational efficiency of the FFT becomes...
854
Basic signals of Fourier Transform01:07

Basic signals of Fourier Transform

The Fourier Transform is a pivotal mathematical tool in signal processing, enabling the transformation of time-domain signals into their frequency-domain representations. Among the numerous elements within this domain, certain functions like the sinc function, delta function, and exponential signals hold significant importance due to their unique properties and implications.
The sinc function, defined as sinc(x) = sin(πx)/(πx), is particularly notable for its symmetry and behavior at...
862
Cartesian Form for Vector Formulation01:26

Cartesian Form for Vector Formulation

The Cartesian form for vector formulation is a process to calculate  the moment of force using the position and force vectors. The moment of force is defined as the cross-product of these vectors, making it a vector quantity. The Cartesian form of the position and force vectors involves unit vectors, which can be used to express the cross-product in determinant form.
1.1K
Vector Algebra: Method of Components01:08

Vector Algebra: Method of Components

It is cumbersome to find the magnitudes of vectors using the parallelogram rule or using the graphical method to perform mathematical operations like addition, subtraction, and multiplication. There are two ways to circumvent this algebraic complexity. One way is to draw the vectors to scale, as in navigation, and read approximate vector lengths and angles (directions) from the graphs. The other way is to use the method of components.
In many applications, the magnitudes and directions of...
18.7K
Discrete Fourier Transform01:15

Discrete Fourier Transform

The Discrete Fourier Transform (DFT) is a fundamental tool in signal processing, extending the discrete-time Fourier transform by evaluating discrete signals at uniformly spaced frequency intervals. This transformation converts a finite sequence of time-domain samples into frequency components, each representing complex sinusoids ordered by frequency. The DFT translates these sequences into the frequency domain, effectively indicating the magnitude and phase of each frequency component present...
827
Parseval's Theorem for Fourier transform01:15

Parseval's Theorem for Fourier transform

Parseval's theorem is a fundamental principle in signal processing that enables the calculation of a signal's energy in either the time domain or the frequency domain. This theorem is pivotal in demonstrating energy conservation between these two domains, ensuring that the computed energy value remains consistent regardless of the domain of analysis.
To understand Parseval's theorem, it is essential to first comprehend how signal energy is typically calculated. When considering a...
2.0K