补充文本引导的注意力为零射击对手的稳定性
相关实验视频
概括
研究人员开发了文本引导注意力为零射击稳定性 (TGA-ZSR) 和辅助文本引导注意力 (Comp-TGA),以提高视觉语言模型对抗对手攻击的稳定性. 这些方法显著提高了对清洁和对抗性示例的零射击准确性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 像CLIP这样的预训练有素的视觉语言模型在零射击任务中表现出色,但易受对抗性示例的影响.
- 在这些模型中,对抗性扰动可以改变文本引导注意力机制.
- 现有的方法在提高强度的同时努力保持概括性.
研究的目的:
- 提高视觉语言模型的对抗性强度,而不牺牲概括性.
- 解决在对抗场景中注意力集中在不相关特征的问题.
- 在具有挑战性的数据集中提高零射击强大的准确性.
主要方法:
- 拟议的文字引导注意力为零射击强度 (TGA-ZSR) 与本地注意力改进和全球注意力约束模块.
- TGA-ZSR将注意力地图从对抗和清洁的例子对齐,并将注意力限制在清洁数据上.
- 引入了辅助文本引导注意力 (Comp-TGA),集成了类和非类提示引导注意力,以实现全面的前景表示.
主要成果:
- TGA-ZSR在零射击强大的精度方面取得了显著的改进.
- 通过使用互补的注意力机制,Comp-TGA进一步提高了稳健性.
- 16个数据集的实验表明,TGA-ZSR和Comp-TGA相对于最先进的方法实现了9.58%和11.95%的改进.
结论:
- 在视觉语言模型中,TGA-ZSR和Comp-TGA是提高对抗性强度的有效策略.
- 互补的注意力机制为强大的零射击学习提供了一个有希望的方向.
- 提出的方法保持了概括性,同时显著提高了对抗性示例的性能.
相关概念视频
02:07Improving Translational Accuracy
15.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.3K
02:07Improving Translational Accuracy
3.7K
3.7K
01:25The Anchoring-and-Adjustment Heuristic
7.8K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.8K
01:14Avoidance Learning and Learned Helplessness
2.8K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.8K