基于影响函数的第二阶段道修剪:评估修剪的真损失变化是可能的,而不需要重新训练
概括
这项研究引入了一种用于神经网络中通道修剪的新方法. 它准确地估计了无需重新训练的性能损失,从而使更可靠的通道选择成为有效的模型压缩.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 频道修剪对于高效的神经网络压缩至关重要.
- 目前的方法依赖于缓慢的再培训或不准确的性能损失估计.
研究的目的:
- 开发一种技术来评估真正的损失变化,而不需要再培训.
- 为了实现可靠和自信的道选择进行修剪.
主要方法:
- 通过使用影响函数,推导出使用影响函数的真损失变化闭式估计器.
- 重用影响功能从可靠的统计数据来评估对真实损失变化的影响.
- 开发了一种基于同时评估道重要性的新型全球道修剪算法.
主要成果:
- 拟议的算法显著优于竞争的道修剪方法.
- 在图像分类和对象检测任务上证明有效.
- 展示了评估切割的真损失变化而不需要再培训的可能性.
结论:
- 开发的技术允许可靠的道修剪,而不需要计算上昂贵的再培训.
- 这一发现为神经网络修剪范式开辟了新的途径.
- 该方法为模型压缩提供了更有效,更准确的方法.
相关概念视频
Reducing Line Loss
150
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
150
Survival Tree
79
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
79
Truncation in Survival Analysis
193
Truncation in survival analysis refers to the exclusion of individuals or events from the dataset based on specific criteria related to the time of the event. This exclusion can happen in two primary forms: left truncation and right truncation.
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
193
Residuals and Least-Squares Property
7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K
Propagation of Uncertainty from Random Error
677
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
677
Propagation of Uncertainty from Systematic Error
516
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
516


