分数张力反复单位 (fTRU):具有长期记忆力的稳定预测模型
IEEE transactions on neural networks and learning systems
|December 15, 2023
概括
我们为高级循环神经网络 (RNN) 引入了分数张量递归单元 (fTRU). 这种模型增强了长期记忆和序列任务的稳定性,在预测方面超过了现有的RNN.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 动态系统 动态系统
背景情况:
- 张量递归模型是利用张量产物的非线性动态系统.
- 现有的模型对长期记忆和对序列任务的稳定性进行了有限的研究.
- 先进的循环神经网络 (RNN) 经常使用这些模型.
研究的目的:
- 为改进序列任务提出一个分数张量递归模型 (fTRU).
- 为了更好的可学习性,将张量度从离散域扩展到连续域.
- 为了平衡长内存特性与模型稳定性.
主要方法:
- 开发了一个分数张量递归单位 (fTRU).
- 从离散域到连续域的扩展张量度.
- 理论上分析了记忆和稳定性之间的权衡.
主要成果:
- 拟议的fTRU在预测任务中实现了具有竞争力的性能.
- 证明了长期记忆和稳定的动态行为.
- 实验结果显示,与各种先进的RNN相比,它具有优势.
结论:
- 在循环神经网络中,fTRU有效地平衡了长期记忆和稳定性.
- 分数扩展使不同数据集的有效学习成为可能.
- 这种模型为序列建模和预测提供了有希望的进步.
相关概念视频
01:19Rectangular and Triangular Pulse Function
708
The unit rectangular pulse function is mathematically represented by a rectangular function centered at the origin with a height of one unit. This function is defined by two parameters: T, which specifies the center location of the pulse along the time axis, and τ, which determines the pulse duration.
For example, consider a rectangular pulse with a 5V amplitude, a 3-second duration, and centered at t=2 seconds. This pulse can be expressed using the rectangular function, written as,
For example, consider a rectangular pulse with a 5V amplitude, a 3-second duration, and centered at t=2 seconds. This pulse can be expressed using the rectangular function, written as,
708
01:03Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
01:20Discrete-Time Fourier Series
276
The Discrete-Time Fourier Series (DTFS) is a fundamental concept in signal processing, serving as the discrete-time counterpart to the continuous-time Fourier series. It allows for the representation and analysis of discrete-time periodic signals in terms of their frequency components. Unlike its continuous counterpart, which utilizes integrals, the calculation of DTFS expansion coefficients involves summations due to the discrete nature of the signal.
For a discrete-time periodic signal x[n]...
For a discrete-time periodic signal x[n]...
276
01:11Residuals and Least-Squares Property
7.4K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.4K
01:23Linear time-invariant Systems
262
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
262
01:24BIBO stability of continuous and discrete -time systems
399
System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
399