重复的对比学习增强了Mamba在时间序列预测中的选择性
Wenbo Yan1, Hanzhong Cao2, Ying Tan3
1School of Intelligence Science and Technology, Peking University, Beijing, China; Computational Intelligence Laboratory, Beijing, China.
概括
重复对比学习 (RCL) 增强了Mamba模型的时间序列预测. 该框架提高了对关键数据点和噪声抑制的关注度,从而实现了最先进的长序列预测性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 时间序列分析时间序列分析
背景情况:
- 长序列预测是时间序列预测的一个重大挑战.
- 基于Mamba的模型显示出希望,但在选择性聚焦和噪音抑制方面存在局限性.
- 这些局限性源于Mamba架构固有的选择能力.
研究的目的:
- 引入重复对比学习 (RCL),一个新的预培训框架.
- 增强Mamba模型的选择性能力,以改善时间预测.
- 在长序预测任务中提高基于Mamba的模型的性能.
主要方法:
- 开发了一个代币级别的对比预训练框架 (RCL).
- RCL预训练一个单一的Mamba块,以加强选择性能力.
- 转移预训练的参数来初始化各种骨干模型中的Mamba块.
- 使用高斯噪声和对比学习策略的序列增强.
主要成果:
- RCL 始终提高了脊柱 Mamba 模型的性能.
- 在长序列预测任务中取得了最先进的结果.
- 与现有方法相比,表现出优越的性能.
- 提出了新的指标来量化Mamba的选择性能力.
结论:
- 重复对比学习 (RCL) 有效地提高了Mamba的选择能力.
- 在时间序列预测中,RCL显著提高了时间预测性能.
- 拟议的框架为长序列预测挑战提供了强大的解决方案.
相关概念视频
Observational Learning
802
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
802
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Prediction Intervals
3.1K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
3.1K

