支持视觉语言模型的少数射击推断与混的剪裁知识提示
Jiangmeng Li1, Wenyi Mo2, Fei Song3
1National Key Laboratory of Space Integrated Information System, Institute of Software Chinese Academy of Sciences, Beijing, China.
概括
本研究介绍了Confounder修剪的知识提示符 (CPKP),这是视觉语言模型的新方法. 通过将语义信息纳入提示函数,CPKP增强了少数射击推理,优于现有方法.
科学领域:
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
- 人工智能的人工智能
背景情况:
- 视觉语言模型将图像文本数据对准为开放式视觉概念.
- 目前的提示方法使用固定的或可学习的提示来弥合预训练和推断的差距.
- 改善推理性能的最佳提示设计仍然不清楚.
研究的目的:
- 澄清视觉语言模型提示中的语义信息的重要性.
- 解决现有的提示方法的局限性,这些提示方法未充分利用文本标签语义.
- 提出一种用于生成语义丰富提示的自动化方法.
主要方法:
- 拟议的Confounder-pruned知识提示符 (CPKP),是一种知识意识的快速学习方法.
- 利用本体知识图来从文本标签中提取语义信息.
- 实施了一种双层混修剪程序 (图表层和特征层) 来完善语义信息.
主要成果:
- 在一些射击推断场景中,CPKP证明了它的有效性.
- 与手动和可学习的提示方法相比,CPKP仅用两次射击就实现了更高的性能.
- 平均而言,CPKP的表现比手动提示方法高4.64%,可学习提示方法高1.09%.
结论:
- 通过知识图表将丰富的语义信息纳入知识图表,显著提高了快速性能.
- 对于视觉语言任务,CPKP提供了一种有效的,自动化的方法来提示工程.
- 拟议的方法在视觉语言模型中提升了少数射击学习能力.
更多相关视频
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.6K
09:27Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
9.9K
相关概念视频
Hindsight Biases
3.4K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
3.4K
Confirmation Biases
5.4K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
5.4K
Associative Learning
285
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
285
Vision
52.9K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.9K
Improving Translational Accuracy
2.5K
2.5K
Strategies for Assessing and Addressing Confounding
82
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
82
