精细的视觉文本提示符驱动的自我训练,用于开放的词汇对象检测.
概括
这项研究引入了一种新的细粒度视觉文本提示驱动的自我训练方法,用于开放的词汇对象检测. 该方法增强视觉语言模型 (VLMs) 以更好的实例本地化,并在未见的类中获得最先进的结果.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 视觉语言模型 (VLMs) 在零射击分类方面表现有前途.
- 将VLM扩展到对象检测通常使用自训练,用于未见类的伪标签.
- 由于全球图像文本预训练,当前的VLM缺乏对对象实例检测至关重要的细粒度对齐.
研究的目的:
- 为开放词汇对象检测 (VTP-OVD) 提出一个细粒度的视觉文本提示驱动的自我训练范式.
- 为了增强VLM的细粒度对齐能力,用于对象检测任务.
- 为了提高从未见过的类中检测对象的性能.
主要方法:
- 引入了一个细粒度的视觉文本提示调整阶段,以改善VLM对齐.
- 采用可学习的文本提示,用于辅助密集的像素预测任务,以实现细粒度的对齐.
- 开发了一个视觉提示模块,为视觉分支提供预先任务信息,以便更好地适应.
主要成果:
- 拟议的VTP-OVD方法显著提高了VLM中的细粒度对齐.
- 在开放词汇对象检测方面取得了最先进的性能.
- 展示了强大的结果,包括COCO数据集的未见类的31.5%mAP.
结论:
- 细粒度的快速调整对于改善VLM在开放词汇对象检测中是有效的.
- VTP-OVD范式提供了一种强大的方法来检测来自未见类的对象.
- 这项工作提升了VLM在复杂的下游视觉任务方面的能力.
相关概念视频
Observational Learning
210
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
210
Force Classification
1.3K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.3K
Introduction to Learning
472
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
472
Vision
53.6K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.6K


