在测试时间对视觉语言模型进行任务对实例快速学习
概括
本研究介绍了视觉语言模型 (VLMs) 的任务对实例快速学习 (TIPPLE),仅使用未标记的数据. 提普尔通过利用任务级和实例级的知识来提高快速学习,以更好地适应.
科学领域:
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 快速学习通过调整可训练的令牌来调整预先训练的视觉语言模型 (VLM).
- 现有的方法往往需要额外的注释数据进行培训.
- 测试时间提示学习方法通常学习每个样本的提示,忽视任务级别的知识.
研究的目的:
- 为VLMs开发一种新的测试时间快速学习方法,只使用未标记的数据.
- 为了提高VLM适应性,利用任务级和实例级知识.
- 解决依赖于注释数据或单个样本快速学习的现有方法的局限性.
主要方法:
- 提出任务对实例的快速学习 (TIPPLE),这是一个两阶段的培训策略.
- 使用在线伪标签范式与辅助文本分类任务.
- 将多样性规范化术语纳入以任务为导向的快速学习.
- 将学习的任务级提示符与可调节的余数结合起来,例如实例级适应.
主要成果:
- 在15个下游数据集上,TIPPLE表现出卓越的性能.
- 与最先进的方法相比,实现了平均1.87%的改进.
- 使用ViT-B/16视觉骨干进行验证.
结论:
- 通过整合任务和实例级知识,TIPPLE有效地通过使用未标记的测试数据来适应VLM.
- 拟议的方法优于现有的测试时间快速学习方法.
- 开源代码促进了进一步的研究和应用.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.7K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.4K
相关概念视频
Improving Translational Accuracy
8.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
8.5K
Associative Learning
273
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
273
Language Development
293
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
293
