在对抗性深度神经网络中,简单压缩的普遍性
Yang Cao1,2,3,4, Yanbo Chen1,2,3,4, Weiwei Liu1,2,3,4
1School of Computer Science, Wuhan University, Wuhan 430072, China.
概括
神经崩 (NC) 显示神经网络捕获数据表示,形成一个简单的结构. 敌对训练压缩了这种简单结构,压缩随扰动半径的增加而增加,为网络稳定性提供了洞察力.
科学领域:
- 深度学习理论 深度学习理论
- 机器学习 稳固性 机器学习 稳固性
- 神经网络分析 神经网络分析
背景情况:
- 神经崩 (NC) 描述了一个现象,其中神经网络输出为类内样本汇聚,而类间样本形成一个简单的等角紧框架 (ETF).
- 了解神经网络内部的内在特性和几何结构对于提高它们的性能和稳定性至关重要.
研究的目的:
- 调查对抗训练对神经崩中观察到的简单ETF结构的影响.
- 在对抗条件下识别和描述潜在的"简单压缩"现象.
- 开发一个理论框架来解释神经表征中观察到的几何变化.
主要方法:
- 跨多种模型和数据集进行实证分析,观察简单ETF的几何尺寸.
- 系统地应用与不同扰动半径的对抗训练.
- 开发一个理论框架来解释观察到的简单压缩现象.
主要成果:
- 在对抗训练下的神经崩中发现了一种新的"简单压缩"现象.
- 简单的ETF的几何尺寸可以通过对抗训练明显减少.
- 简单压缩的程度与敌对攻击的扰动半径正相关.
结论:
- 敌对训练会导致神经表征中几何结构的压缩,神经崩证明了这一点.
- 这些发现提供了更深入的了解在对抗条件下神经网络行为的神经网络.
- 已建立的理论框架为神经网络的稳定性和神经崩的机制提供了洞察力.
相关概念视频
Reducing Line Loss
130
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
130
Improving Translational Accuracy
8.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
8.5K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
28
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
28
Force Classification
1.0K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.0K
Multi-input and Multi-variable systems
88
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
88
Downsampling
110
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
110


