ProtoCLIP:原型的对比语言图像预训练

相关实验视频

Updated: Jul 9, 2025

Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms

Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms

Published on: February 8, 2019

Delong Chen, Zhao Wu, Fan Liu

IEEE transactions on neural networks and learning systems

|December 4, 2023

概括

原型对比语言图像预训练 (ProtoCLIP) 增强了表示分组,以提高效率和稳定性. 这种方法可以提高下游任务的性能,并且与CLIP相匹配.

科学领域:

计算机科学计算机科学
人工智能的人工智能
机器学习机器学习

背景情况:

对比性语言图像预训练 (CLIP) 擅长将学习到的表征转移到下游任务中.
CLIP的InfoNCE目标是对准正图像-文本对并分离负图像,间接分组相似的表示.
在CLIP中通过模式内点观察到一个潜在的代表性分组效应.

研究的目的:

引入原型对比语言图像预训 (ProtoCLIP) 以提高表示分组的效率和稳定性.
改善多式联络模式中的转移学习能力.
为了解决代表性学习中的模式差距.

主要方法:

ProtoCLIP 建立了在图像和文本空间之间进行原型级别的区分,以实现高效的知识传输.
原型反向翻译 (PBT) 将表示分组与对齐脱,使得在很大的模式差距下学习成为可能.
在线插曲式培训策略允许扩展到无限数据,并结合外部知识来源.

主要成果:

在概念捕捉数据集上,ProtoCLIP实现了 +5.81%的ImageNet线性探测改进和 +2.01%的ImageNet零射击分类改进.
在YFCC-15M数据集中,ProtoCLIP使用只有33%的培训时间,与CLIP的表现相匹配.
提出的方法表明,尽管存在显著的模式差距,但有效地学习了有意义的表示.

结论:

ProtoCLIP提供了一种更有效,更强大的方法来进行对比性语言形象预训练.
通过PBT将分组与对齐脱是处理模式差距和整合外部知识的关键.
ProtoCLIP为现有的预训练方法提供了可扩展和有效的替代方案,减少了训练时间和提高了性能.

更多相关视频

Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition

Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition

Published on: July 13, 2019

Portable Intermodal Preferential Looking IPL: Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism

Portable Intermodal Preferential Looking IPL: Investigating Language Comprehension in Typically Developing Toddlers and Young Children with Autism

Published on: December 14, 2012

相关概念视频

Stereotype Content Model

Stereotype Content Model

The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...