端到端的开放词汇视频视觉关系检测使用多模式提示
概括
本研究引入了开放词汇视频视觉关系检测的统一框架,增强了对象之间的看不见关系的检测. 这种新的方法集成了轨迹检测和关系分类,改进了对新对象类别的概括.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 目前的开放词汇视频视觉关系检测方法依赖于预训练的轨迹检测器,限制了对新型对象类别的概括.
- 这种依赖导致在遇到看不见的对象和关系时性能下降.
研究的目的:
- 开发一个端到端的开放词汇框架,统一对象轨迹检测和关系分类.
- 提高视频视觉关系检测系统对新型对象类别和关系的概括能力.
主要方法:
- 提出一种基于关系的开放词汇轨迹探测器,使用基于查询的变压器解码器和蒸的CLIP视觉编码器.
- 整合关系查询和辅助损失,在轨迹检测过程中明确感知对象关系.
- 开发一个开放的词汇关系分类器,采用多模式提示方法 (时空视觉和视觉引导的语言提示),以适应CLIP.
主要成果:
- 拟议的框架证明了对VidVRD和VidOR数据集的有效性.
- 该方法在具有挑战性的跨数据集场景中显示出强大的概括能力.
结论:
- 统一框架成功地解决了开放词汇视频视觉关系检测现有方法的局限性.
- 开发的关系感知检测器和分类器显著增强了新型关系的检测,并改善了概括.
相关概念视频
Observational Learning
101
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
101
Sign Test for Matched Pairs
80
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
80
Visual Agnosia
166
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
166
Vision
52.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.3K
Purposive Learning
90
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
90
Associative Learning
246
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
246


