校准多模式表示:在没有注释的情况下追求群体稳定性
Chenyu You1, Yifei Min1, Weicheng Dai1
1Yale University.
概括
本研究引入了一种轻量级方法来微调CLIP模型,减少对虚假特征的依赖,以便在不需要组标签的情况下更好地概括. 这种方法提高了现实世界应用的模型稳定性.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 预训练的视觉语言模型,如CLIP是有效的,但面临着计算成本,专业化和依赖虚假特征的挑战.
- 假特征与标签相关,但没有因果关系,阻碍了模型的概括和现实世界的部署.
- 现有的减轻虚假特征的方法通常需要识别这些特征,缺乏实际使用的最终保证.
研究的目的:
- 探索减轻CLIP模型中虚假特征的依赖,而不需要组注释的方法.
- 系统地调查CLIP和CLIP与经验风险最小化 (ERM) 中假相关的存在.
- 提出和验证一种轻量级表示校准方法,用于微调CLIP.
主要方法:
- 经过验证,最后一层再训练在预先训练的CLIP上提高了小组的强度,遵循深度特征重权 (DFR) 原则.
- 开发了一种新的轻量级表示校准方法,用于微调CLIP.
- 使用预先训练的CLIP生成了一个校准集,并使用对比学习校准样本表示,所有这些都没有组标签.
主要成果:
- 在CLIP模型中显著减少对虚假特征的依赖.
- 在各种基准的模型通用化方面取得了实质性的改进.
- 通过广泛的实验和可视化验证了拟议的表示校准方法的有效性.
结论:
- 拟议的轻量级表示校准方法有效地减轻了CLIP中的虚假特征依赖.
- 这种方法增强了模型的概括性和稳定性,使其更适合于现实世界的应用.
- 该方法为微调视觉语言模型提供了一个实用的解决方案,而不需要昂贵的组注释.
相关概念视频
Multi-input and Multi-variable systems
96
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
96
Calibration Curves: Linear Least Squares
1.2K
A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
For data that follow a straight line, the standard method for fitting is the linear...
1.2K
Calibration Curves: Correlation Coefficient
1.5K
In a linear calibration curve, there is a value called the calibration coefficient, denoted by 'r,' which measures the strength and the direction of association between two variables. The correlation coefficient value ranges from −1 to +1. A value of +1 indicates a perfect positive linear correlation, −1 denotes a perfect negative correlation, and 0 implies no correlation between the two variables. A positive correlation value establishes that as one variable increases, the...
1.5K
Modeling and Similitude
245
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
245
Generalization, Discrimination, and Extinction
440
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
440
Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K


