DGPrompt:用于视觉语言模型的双导向提示生成
Tai Zheng1, Zhen-Duo Chen1, Zi-Chao Zhang1
1School of Software, Shandong University, 1500 Shunhua Road, Jinan 250101, China.
概括
双导向提示生成 (DGPrompt) 通过改善跨模式对齐和保留一般知识来增强CLIP. 这种方法可以提高视觉识别任务的性能,特别是在有限的数据的情况下.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 自然语言处理自然语言处理.
背景情况:
- 通过可学习提示,CLIP模型通过可学习提示实现强的性能.
- 现有的方法受到有限的模式间相互作用和层次上下文的影响,阻碍了对齐.
- CLIP对提示的敏感性导致了对看不见的类的过拟合和减少泛化.
研究的目的:
- 提出双导向提示生成 (DGPrompt) 以改善视觉-文本对齐.
- 通过防止遗忘CLIP的一般知识来提高概括能力.
- 为了解决现有的CLIP快速调整方法的局限性.
主要方法:
- DGPrompt利用视觉和文本提示之间的相互指导进行嵌入提取.
- 整合了一个保留模块,以限制快速调整和保存一般知识.
- 该方法侧重于有效地对齐视觉和文本表示空间.
主要成果:
- 与基线CLIP和最先进的MaPLe相比,DGPrompt显示出更高的性能.
- 在11个数据集的总和平均值上获得了7.84%和0.99%的绝对收益.
- 在基础到新类的概括和少量学习场景中表现出有效性.
结论:
- DGPrompt成功地促进了视觉和文本空间之间的对齐.
- 该方法有效地保留了一般知识,改善了对未见类的概括.
- DGPrompt为视觉语言模型的基于提示的学习提供了显著的进步.
更多相关视频
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
205
06:15Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism
Published on: October 3, 2018
7.6K
相关概念视频
Improving Translational Accuracy
2.5K
2.5K
Vision
52.2K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.2K
Multi-input and Multi-variable systems
86
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
86
Force Classification
1.0K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.0K
Depth Perception and Spatial Vision
453
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
453
Genetic Lingo
98.4K
Overview
98.4K
