时间先验的自我注意:我们可以从时间的箭头中学到更多吗?
Kyung Geun Kim1, Byeong Tak Lee2
1VUNO Inc., Seoul, Republic of Korea.
Frontiers in artificial intelligence
|August 21, 2024
概括
这项研究引入了一种新的方法来改进时间序列数据的基于注意力的模型. 通过应用自适应内核,该模型更好地捕捉短期时间依赖,增强电子健康记录的预测.
科学领域:
- 机器学习 机器学习
- 时间序列分析时间序列分析
- 计算科学 计算科学
背景情况:
- 自然现象表现出短期和长期的时间依赖,受时间流的影响.
- 对于更近的时间,事件的相互关系更强.
- 基于注意力的模型因数据要求和缺乏时间偏差编码而在时间序列中与短期依赖性作斗争.
研究的目的:
- 开发一种有效的方法,让注意层对短期时间偏差进行编码.
- 提高基于时间序列数据的基于注意力的模型的性能,具有固有的时间依赖性.
主要方法:
- 提出了一种方法,将可学习,适应的内核直接应用于注意力矩阵.
- 利用电子健康记录 (EHR) 数据集用于具有已知的时间依赖性的预测任务.
- 与最先进的模型对比,对方法进行了评估.
主要成果:
- 拟议的方法显著提高了注意层编码短期时间偏差的能力.
- 在使用EHR数据的各种预测任务中取得了卓越的分类结果.
- 在大多数任务和数据集上表现优于现有的表现最好的模型.
结论:
- 适应性内核方法是一种简单而有效的解决方案,用于改善时间序列中的注意力模型.
- 这种方法解决了学习短期时间依赖性的局限性,特别是在像EHR这样的复杂数据集中.
- 在捕捉时间规律方面表现出卓越的性能,这对于准确的预测至关重要.
相关概念视频
Hindsight Biases
3.4K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
3.4K
Purposive Learning
106
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
106
Cognitive Learning
233
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
233
Framing Effects
7.3K
Information is everywhere and its presentation—such as how and when items are presented—can impact our perceptions and decisions surrounding the info. This broad concept umbrellas framing effects—influences that occur due to the way information is framed in its appearance, whether it’s purely the order or the specific wording of a message. Let’s take a look at numerous ways in which two versions of something can objectively say the same thing, yet we respond in...
7.3K
Steps in the Modeling Process
193
Albert Bandura's theory of observational learning identifies four critical processes: attention, retention, motor reproduction, and reinforcement or motivation.
Attention is the first necessary component for observational learning. It involves focusing on what the model is doing and saying. For example, if you decide to take a drawing class to enhance your skills, you need to pay close attention to the instructor's words and hand movements. The characteristics of the model significantly...
Attention is the first necessary component for observational learning. It involves focusing on what the model is doing and saying. For example, if you decide to take a drawing class to enhance your skills, you need to pay close attention to the instructor's words and hand movements. The characteristics of the model significantly...
193
Observational Learning
155
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
155


