睡眠调整:神经增强的快速调整,用于对抗强大的视觉语言模型
IEEE transactions on pattern analysis and machine intelligence
|January 30, 2026
概括
多模式对抗性即时调整 (NAP-Tuning) 的神经增强器框架增强了视觉语言模型的安全性. 它净化了内部层面的功能,显著提高了对抗攻击的强度.
科学领域:
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
- 机器学习安全 机器学习安全
背景情况:
- 视觉语言模型 (VLMs) 在联合视觉文本理解方面表现出色,但易受对手攻击.
- 像Adversarial Prompt Tuning (AdvPT) 这样的现有防御提高了稳定性,但可以增强.
- 敌对干扰对VLMs构成重大安全风险.
研究的目的:
- 引入神经增强器框架用于多模式对抗式快速调整 (NAP-Tuning).
- 通过多模式,多层特征净化,增强VLM中的对抗性强度.
- 开发一种适应性防御机制,用于识别和纠正对抗性干扰.
主要方法:
- 开发了一个全面的多模式 (文本和视觉) 和多层提示框架 (NAP-Tuning).
- 使用TokenRefiners实现神经增强器方法,通过剩余连接进行特征级净化.
- 在各种数据集和攻击类型中进行了实验,包括AutoAttack.
主要成果:
- NAP-Tuning显著优于现有的对抗性稳定性方法.
- 在AutoAttack下实现了与基线相比显著的改进 (32.3%在ViT-B16,31.3%在ViT-B32).
- 保持了竞争力的清洁准确性,同时增强了对手的防御能力.
结论:
- 内部特征级干预是有效的,可以快速调整对抗性强度.
- NAP-Tuning通过纠正嵌入空间中的扰动来提供适应性防御.
- 这种方法超出了输入侧对齐,用于更强大的VLM.
相关概念视频
Problem-Solving: Tuning of a Guitar String
1.1K
In the case of stringed instruments like the guitar, the elastic property that determines the speed of the sound produced is its linear mass density or the mass per unit length. This is simply called the linear density. If the string's linear density is constant along the string, then the linear density is simply the total mass divided by the total length.
The string's wave speed can be regulated by varying the linear density. Tension is the other property that determines the speed of...
The string's wave speed can be regulated by varying the linear density. Tension is the other property that determines the speed of...
1.1K
Vision
60.0K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
60.0K
Language
910
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
910
Color Vision
1.5K
Color perception begins in the retina, the light-sensitive layer at the back of the eye. Two main theories explain how colors are seen: the trichromatic theory and the opponent-process theory. The trichromatic theory, proposed by Thomas Young in 1802 and extended by Hermann von Helmholtz in 1852, suggests that color vision is based on three types of cone receptors in the retina. These cones are sensitive to different but overlapping ranges of wavelengths corresponding to red, blue, and green.
1.5K
Components of Language
820
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
820
Language Development
912
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
912


