S$^{2}$2O:通过二次权重统计来增强对手训练.
概括
这项研究引入了第二阶段统计优化 (S$^{2}$2O),以提高深度神经网络的稳定性. S$^{2}$2O通过优化权重统计数据来改善对手训练,从而提高了概括性和安全性.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 优化理论 优化理论
背景情况:
- 敌对训练可以提高深度神经网络 (DNN) 对干扰的强度.
- 当前的方法通常依赖于模型权重的统计独立性的不现实的假设.
- 像SGD这样的梯度下降方法是优化DNN权重的标准.
研究的目的:
- 提出一种新的方法来增强使用二级统计优化 (S$^{2}$2O) 的对抗训练.
- 在PAC-贝叶斯框架中放松权重的统计独立性假设.
- 为了获得一个改进的PAC-贝叶斯强大的概括.
主要方法:
- 将模型权重视为随机变量来进行优化.
- 在模型权重上开发和应用二级统计优化 (S$^{2}$2O).
- 在PAC-贝叶斯分析中放松统计独立假设.
主要成果:
- 导出了一个改进的PAC-贝叶斯强大的概括.
- 证明优化二级权重统计数据可以收紧泛化界限.
- 经验证实,S$^{2}$2O增强了DNN的稳定性和通用性.
结论:
- S$^{2}$2O提供了一种原则性的方法来改善对抗训练.
- 该方法提高了DNN的稳定性和通用性.
- S$^{2}$2O有效地补充了现有的最先进的对抗训练技术.
相关概念视频
Weighted Mean
6.2K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
6.2K
Randomized Experiments
8.9K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
8.9K
Statistical Analysis: Overview
15.0K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
15.0K
One-Way ANOVA: Unequal Sample Sizes
6.6K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
6.6K
Testing a Claim about Standard Deviation
2.9K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.9K
Associative Learning
1.3K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.3K
