量子化卷积神经网络 在扰乱下强度
Jack Langille1, Issam Hammad1, Guy Kember1
1Department of Engineering Mathematics and Internetworking, Dalhousie University, Halifax, Nova Scotia, Canada.
F1000Research
|May 1, 2025
概括
量子化卷积神经网络 (CNN) 在输入扰动下保持性能,相对误差低. 库尔巴克-利布勒分歧显示了最小的变化,除了在VGG-16和SqueezeNet1_1.1.上布朗噪声效应.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 机器学习模型由于尺寸和操作而面临计算限制.
- 量子化通过使用精度较低的整数来减少模型大小和计算需求.
- 现有的研究证实量子化模型与完全精确的性能相匹配,但缺乏在输入扰动下进行分析.
研究的目的:
- 在扰乱输入条件下研究8位量化卷积神经网络 (CNN) 的性能.
- 解决关于量子化模型在噪音环境中的稳定性的文献上的差距.
- 评估输入扰动对模型准确性和输出分布相似性的影响.
主要方法:
- 研究了三个CNN (ResNet-18,VGG-16,SqueezeNet1_1) 在浮点和8位量化形式.
- 应用了各种不同强度的噪声模式来模拟输入.
- 使用top-1/top-5精度,F1分数测量性能,并引入Kullback-Liebler分歧来评估输出分布变化.
主要成果:
- 量子化模型在所有测试的扰动中,与完全精确的对应模型相比,呈现出始终较低的相对误差.
- 库尔巴克-利布勒分歧仍然与未受到干扰的测试相比,表明决策的相似性稳定.
- 在VGG-16和SqueezeNet1_1的输出分布中观察到显著的差异,特别是在布朗噪声干扰下.
结论:
- 8位量子化的CNN显示出强度,即使受到输入噪声的影响,也保持了性能的一致性.
- 库尔巴克-利布勒分歧是量化对压力下模型输出相似性的量化影响的一个有价值的指标.
- 虽然通常是强大的,但特定的模型和噪声类型 (例如,布朗噪声) 需要进一步调查潜在的漏洞.
相关概念视频
00:59Propagation of Uncertainty from Random Error
500
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
500
01:20Convolution Properties I
118
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
118
01:17Convolution Properties II
145
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
145
01:10Propagation of Uncertainty from Systematic Error
345
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
345
01:24Convolution: Math, Graphics, and Discrete Signals
201
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
201
01:28Accuracy, limits, and approximation
409
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
409