没有对抗防御的对抗防御:通过删除实例级主要组件来提高语言模型的稳定性
Yang Wang1,2, Chenghao Xiao3, Yizhi Li4
1The University of Manchester, UK. yang.wang-27@postgrad.manchester.ac.uk.
概括
本研究引入了一种新型模块,以提高预训练语言模型 (PLM) 对抗对方攻击的稳定性. 该方法可以提高模型的安全性,而不会增加计算成本或改变训练数据.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 机器学习安全 机器学习安全
背景情况:
- 预训练语言模型 (PLM) 在自然语言处理方面取得了重大进展.
- PLM容易受到对抗性攻击,从而损害了它们在现实世界的可靠性.
- 现有的防御机制往往涉及计算上昂贵的对抗训练或数据增强.
研究的目的:
- 开发一个高效的附加模块,以提高PLM的对抗性稳定性.
- 在不依赖传统防御或修改训练数据的情况下,减轻对抗性漏洞.
- 为了保持模型性能和概括性,同时提高稳定性.
主要方法:
- 提出了一个新的模块,从嵌入空间中删除实例级主要组件.
- 将嵌入式转换为近似的高斯属性,减少对扰动的易感性.
- 调整嵌入式分布以尽量减少对决策边界的对抗性噪声影响.
主要成果:
- 拟议的方法显著提高了八个基准数据集的对抗性稳定性.
- 在对抗性攻击之前,保持与基线模型相似的准确性.
- 在增强的稳定性和通用化能力之间展现出平衡的权衡.
结论:
- 附加模块提供了一种有效且计算效率高的解决方案,用于提高PLM对抗性稳定性.
- 该方法通过改变嵌入空间来增强模型安全性,而不是通过改变培训程序.
- 这种方法为在安全敏感的应用程序中部署强大的PLM提供了一种实用的方法.
更多相关视频
相关概念视频
Language Development
813
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
813
Components of Language
737
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
737
Principal Moments of Area
1.7K
In mechanics, the product of inertia and moments of inertia of area help to calculate the stability and performance of various structures and components. The coordinate transformation relations are used to calculate the moments and products of inertia for an area about the inclined axes. Further, the moments and products of inertia with respect to the principal axes can be determined using the moments and products of inertia about the inclined axes.
The principal moment of inertia axes are the...
The principal moment of inertia axes are the...
1.7K
Improving Translational Accuracy
14.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.0K
Improving Translational Accuracy
3.5K
3.5K
Quantifying and Rejecting Outliers: The Grubbs Test
3.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
3.5K


