RoGAtten:用于多变量时间序列预测的旋转门线性注意力.
Aobo Liang1, Yan Sun1, Xiaohou Shi2
1School of Computer, Beijing University of Posts and Telecommunications, Beijing, 100876, Beijing, China.
概括
本研究介绍了用于网络流量预测的旋转门线性注意力 (RoGAtten). RoGAtten通过整合Mamba-like gating来增强变压器模型,改善长期依赖模型和预测准确度.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 网络工程 网络工程
背景情况:
- 物联网 (IoT) 产生了大量的长期时间序列数据.
- 准确的网络流量预测对于安全和管理至关重要.
- 变压器模型提供高预测准确性,但在注意力机制效率方面存在困难.
研究的目的:
- 加强注意力机制,以改善网络流量预测.
- 开发一种新型模型,平衡表达性和计算效率.
- 利用像Mamba这样的状态空间模型的见解来提高注意力.
主要方法:
- 从理论上证明,带有旋转定位嵌入的线性注意力可以类似于Mamba.
- 设计了一个具有扩展因子的可扩展旋转位置嵌入 (SRoPE) 机制.
- 建议用于多变量时间序列预测的旋转门式线性注意力 (RoGAtten).
- 集成的SRoPE提供序列智能标识符并调整变量间的相互作用.
主要成果:
- SRoPE赋予了类似忘记门的能力,增强了表现力,而不是以前的注意力变体.
- RoGAtten有效地捕捉了多变量时间序列中的系列间依赖关系.
- 在8个现实世界数据集上的实验显示了显著的性能改善.
- 与最先进的方法相比,RoGAtten减少了3.85%的平均平方误差 (MSE) 和1.71%的平均绝对误差 (MAE).
结论:
- 在网络流量预测中,RoGAtten为多变量时间序列预测提供了一种优越的方法.
- 拟议的SRoPE机制增强了模型的适应性和与领域知识的协调性.
- 这项研究为时间序列分析的现有注意力机制提供了一个计算效率高和高度表现力的替代方案.
相关概念视频
Multi-input and Multi-variable systems
394
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
394
Linear time-invariant Systems
874
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
874
Prediction Intervals
3.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
3.3K
Linear Approximation in Time Domain
347
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
347
End Point Prediction: Gran Plot
1.2K
A Gran plot is used to predict the equivalence volume or endpoint of a potentiometric or acid-base titration without reaching the endpoint. Typically, titration data is collected as a function of the titrant's volume up to a point less than the equivalence volume and then transformed into a linear format. The straight line is extended to the x-axis, indicating the necessary titrant volume to achieve the equivalence point.
For potentiometric titration, the Gran plot is created by plotting...
For potentiometric titration, the Gran plot is created by plotting...
1.2K
Sampling Continuous Time Signal
690
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
690

