在深度神经网络中使用詹森-香农和随机森林特征重要性排名的前向传播掉落
Mohsen Heidari1, Mohammad Hossein Moattar2, Hamidreza Ghaffari1
1Department of Computer Engineering, Ferdows Branch, Islamic Azad University, Ferdows, Iran.
概括
本研究引入了深度神经网络的新型动态脱落方法,通过在训练过程中始终识别和保存重要的节点来提高准确性和概括性. 该方法提供了高效的节点重要性计算和有效的过防止.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 深度神经网络 (DNN) 容易过度匹配,阻碍了概括.
- 随机丢失,一种常见的规范化技术,随机删除节点,但可以降低网络准确性.
- 现有的动态丢弃方法与一致的节点重要性计算和计算成本相斗争.
研究的目的:
- 开发一个更有效和高效的DNN的动态退出机制.
- 为了解决与传统动态弃方法相关的不一致性和高计算成本.
- 为了提高DNN的准确性,概括性和节点效率.
主要方法:
- 一种使用随机森林和Jensen-Shannon分歧来计算节点重要性的一次性的新型动态抛弃方法.
- 在前向传播过程中传播节点的重要性,以实现一致的脱落应用.
- 对不同数据集 (MNIST,NorB,CIFAR10,CIFAR100,SVHN,ImageNet) 和DNN架构的现有脱落技术进行评估.
主要成果:
- 与其他弃方法相比,拟议的方法实现了更高的准确性和更好的通用性.
- 它在较少的网络节点上证明了有效性.
- 该方法表现出相似的复杂性和相对于最先进的技术相对较短的融合时间.
结论:
- 新的动态脱落方法提供了一个计算效率高和有效的解决方案,以防止DNN的过拟合.
- 它通过始终保持关键节点来提高模型性能.
- 这种方法代表了深度学习规范化技术的重大进步.
相关概念视频
00:59Propagation of Uncertainty from Random Error
740
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
740
01:19Survival Tree
118
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
118
01:10Propagation of Uncertainty from Systematic Error
565
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
565
01:08Outliers and Influential Points
4.1K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.1K
01:09Random Variables
12.4K
A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
12.4K
01:03Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K