在快速对抗训练中避免灾难性的过拟合,使用适应性相似性步骤大小
Jie-Chao Zhao1, Jin Ding1,2, Yong-Zhi Sun1
1School of Automation and Electrical Engineering & Key Institute of Robotics of Zhejiang Province, Zhejiang University of Science and Technology, Hangzhou, China.
PloS one
|January 8, 2025
概括
快速的对抗式训练方法提高了深度学习模型的稳定性,但可能会导致过拟合. 本研究介绍了ATSS,一种自适应相似度步骤大小方法,可以提高稳定性和准确性,而无需显著的计算成本.
科学领域:
- 深度学习 (Deep Learning) 是一种深度学习.
- 机器学习安全 机器学习安全
- 计算机视觉 计算机视觉
背景情况:
- 对抗性训练对于深度学习模型的稳定性至关重要.
- 快速的对抗训练方法提供了计算效率,但受到有限的对抗示例多样性的影响,导致过拟合和降低稳定性.
- 现有的减轻过拟合的方法在对抗示例强度和整体扰动方面存在局限性.
研究的目的:
- 提出一种新的快速对抗训练方法,即ATSS (自适应相似度步骤大小),以解决当前方法的局限性.
- 提高深度学习模型的稳定性和准确性,防止对抗性攻击,同时防止灾难性的过度装配.
主要方法:
- ATSS涉及将随机噪声添加到清洁样品中,并计算梯度.
- 干扰的步骤大小是根据输入噪声和梯度方向之间的相似性而自适应地确定.
- 使用适应式步骤大小进行培训,生成对抗性示例.
主要成果:
- 在CIFAR-10,CIFAR-100和Tiny ImageNet数据集中的ResNet18和VGG19模型上的实验结果证明了ATSS的有效性.
- 拟议的ATSS方法成功避免了灾难性的过度装配.
- 与现有的快速对抗训练方法相比,ATSS实现了卓越的稳定性和清洁的准确性,而额外的训练成本最小.
结论:
- ATSS提供了一个有效的解决方案,用于快速对抗训练,平衡稳定性和准确性.
- 适应性相似性步骤大小机制是产生多样化和有效的对抗性示例的关键.
- 这种方法提供了一种计算效率高的方法,可以显著提高深度学习模型的弹性.
相关概念视频
Improving Translational Accuracy
2.5K
2.5K
Accuracy, limits, and approximation
439
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
439
Limits to Natural Selection
31.1K
Organisms that are well-adapted to their environment are more likely to survive and reproduce. However, natural selection does not lead to perfectly adapted organisms. Several factors constrain natural selection.
31.1K
Reducing Line Loss
143
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
143
Molecular Weight of Step-Growth Polymers
2.1K
Step growth polymerization involves bi or multifunctional monomers. Bifunctional monomers react to form linear step growth polymers, whereas multifunctional monomers react to form non-linear or branched polymers.
As the step-growth polymerization involves step-wise condensation of monomers, the molecular weight also builds up eventually. Consequently, high molecular weight polymers are obtained at the late stages of the polymerization, where 99% of monomers have been consumed.
The extent of the...
As the step-growth polymerization involves step-wise condensation of monomers, the molecular weight also builds up eventually. Consequently, high molecular weight polymers are obtained at the late stages of the polymerization, where 99% of monomers have been consumed.
The extent of the...
2.1K
Sample Size Calculation
3.2K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.2K


