通过稀疏的知识引导的上下文优化来增强视觉语言的快速调整
1School of Information and Electrical Engineering, Hangzhou City University, Hangzhou 310015, China.
Entropy (Basel, Switzerland)
|March 28, 2025
概括
斯帕斯知识引导的上下文优化 (Sparse-KgCoOp) 改进了视觉语言模型 (VLM) 的提示调整. 这种方法通过减少适应性提示和手工提示之间的差异来增强对新类别的概括性,从而保持核心知识.
科学领域:
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
- 人工智能的人工智能
背景情况:
- 快速调整适合特定任务的视觉语言模型 (VLMs),使用特定任务的令牌.
- 像CoOp这样的现有方法可能导致对未见的类别的概括性不佳,从而掩盖了一般知识.
研究的目的:
- 为了解决VLM提示调整中的基础新奇困境.
- 提高适应性提示对新类别的概括能力.
主要方法:
- 建议Sparse以知识为导向的上下文优化 (Sparse-KgCoOp).
- 使用散散化操作来减少适应性提示和手工提示之间的差异.
- 缩小动态和手动设计的文本嵌入之间的差距.
主要成果:
- 斯帕斯-KgCoOp展示了高效的提示调整.
- 该方法有效地缓解了专业化过程中基本知识的侵蚀.
- 实验表明,对不熟悉的类别进行了改进的概括.
结论:
- Sparse-KgCoOp提供了一种有效的解决方案,用于改进VLM提示调整通用化.
- 该技术保留了基础知识,同时使其能够适应新任务和新类别.
相关概念视频
The Anchoring-and-Adjustment Heuristic
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the $2,000...
Cognitive Learning
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
Purposive Learning
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a bonus...
Optimization Problems
Optimization problems often involve identifying maximum or minimum values under specific constraints. A well-known example is determining the longest horizontal pipe that can be moved around a right-angled corner, where a 3-meter-wide hallway meets a 2-meter-wide hallway. This scenario, common in architectural design and industrial transport, can be understood conceptually through geometric and trigonometric reasoning.To visualize the problem, consider the pipe as a straight line that touches...
Impression Management Techniques IV: Altercasting
Altercasting is a strategic communication technique in which an individual imposes a specific identity or social role onto another person to influence their behavior and shape the interaction. By presuming a role—such as “responsible leader” or “patient person”—altercasting encourages the target to conform to that identity, often aligning their behavior with the expectations associated with the role. The power of this tactic lies in its subtlety; once a role is assigned, it becomes socially...
Methods of Medium Optimization
Optimizing growth media enhances microbial proliferation and maximizes product yield. Statistical experimental design methodologies provide structured and reproducible approaches, offering progressively higher levels of robustness and efficiency.The One-Factor-at-a-Time (OFAT) MethodThe One-Factor-at-a-Time (OFAT) method involves adjusting a single variable while keeping all others constant. However, it cannot detect interactions between variables, often leading to suboptimal outcomes when...


