从一般化边际角度分析对抗性训练的隐含偏见
概括
反对训练通过对齐层重量和最大限度地提高对抗示例的边际来提高深度神经网络的稳定性. 本理论分析解释了这种方法如何提高概括性,并为新的强有力的培训策略提供了洞察力.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 优化理论 优化理论
背景情况:
- 深度神经网络 (DNN) 容易受到对抗的例子的影响.
- 敌对训练提高了DNN的稳定性,但其机制尚不清楚.
研究的目的:
- 从理论和经验上分析对抗训练的隐含偏见.
- 了解对抗性训练如何提高深度神经网络的稳定性.
主要方法:
- 对同质DNN的对抗性训练的理论分析.
- 对于深度线性和一般均网络的一般化边际前景.
- 以统一的方式调查各种对抗性扰动.
主要成果:
- 重量矩阵对齐,在深线性网络中最大限度地提高对抗性示例边缘.
- 参数收与优化最大化边际的KKT点保持一致.
- 使用 L_p 规范进行对抗训练,可以最小化特定线性网络的插值规范.
结论:
- 对抗性培训通过使用对抗性示例来修改决策边界,以提高强度.
- 为现有的实证观测提供理论基础.
- 为开发新的强大培训策略提供了见解.
相关概念视频
Motivational Bias
1
Cognitive bias results from limitations in thinking and information processing, leading to systematic errors in judgment. Conversely, motivational bias stems from personal desires or emotions, causing distortions in perception to align with self-interest. Motivational bias influences how individuals perceive and attribute causes to events, often shaped by personal needs, goals, and self-esteem preservation. This bias can distort judgment, leading to inaccurate assessments of success, failure,...
1
Bias
5.6K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
5.6K
Margin of Error
4.5K
The margin of error is also called the maximum error of an estimate. The margin of error is the maximum possible or expected difference between the observed sample parameter value and the actual population parameter value. For proportion, it is the maximum difference between the value of sample proportion obtained from the data and the true value of population proportion. As the true value of the population parameter is not known, the margin of error is calculated using the sample statistic.
4.5K
Stereotypes, Prejudice, and Discrimination
91.7K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
91.7K
Generalization, Discrimination, and Extinction
826
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
826
Stereotype Content Model
14.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.9K

