走向通用模态跟踪与在线密集的时间令牌学习
概括
我们介绍UM-ODTrack,这是一个通用的视频级模式意识的跟踪模型. 该模型支持单一架构的多种跟踪任务,通过利用时间令牌学习实现最先进的性能.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 多模式跟踪通过整合各种传感器数据 (例如RGB,热,深度,事件) 来提高稳定性.
- 现有的多模式追踪器通常需要特定任务的架构和广泛的独立培训.
- 对于各种跟踪方式的统一方法仍然是一个重大挑战.
研究的目的:
- 开发一个通用的视频级模式意识的跟踪模型 (UM-ODTrack),可适应多个跟踪任务.
- 为了使单个模型架构和参数设置能够处理RGB,RGB+热,RGB+深度和RGB+事件跟踪.
- 通过新的时间令牌学习和跨模式融合,提高追踪性能并减少培训复杂性.
主要方法:
- 视频级采样以捕捉更广泛的时间背景.
- 在线密集的时间令牌协会用于外观和运动传播.
- 带有注意力机制的门式感知器,用于适应性交叉模式表示学习.
- 一次性培训,用于模式可扩展的多任务推理.
主要成果:
- UM-ODTrack在可见和多模式基准上实现了最先进的 (SOTA) 性能.
- 该模型有效地利用以前的框架信息作为未来推断的时间提示.
- 一次性培训计划显著降低了培训负担,同时提高了模型的代表性.
结论:
- 对于各种视频跟踪任务,UM-ODTrack提供了一个统一而高效的解决方案.
- 拟议的方法在不同模式中展示了卓越的性能和概括能力.
- 这项工作推进了多模式视觉跟踪领域,采用了可扩展和有效的方法.
相关概念视频
01:12Observational Learning
314
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
314
01:27Associative Learning
579
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
579
01:22Multi-input and Multi-variable systems
150
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
150
01:18Long-Term Memory
257
Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
257
01:26Per-Unit Sequence Models
116
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
116
01:18Introduction to Learning
533
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
533