分类器深度学习中的交叉是不必要的-ISBE错误就是你所需要的一切
1Faculty of Electronics and Information Technology, Warsaw University of Technology, 00-661 Warszawa, Poland.
Entropy (Basel, Switzerland)
|January 22, 2024
概括
本研究介绍了改进的SoftMax-Before-Entropy (ISBE) 功能,证明交叉计算在深度学习分类器中是多余的. ISBE简化了反向传播,提高了效率而不降低性能.
科学领域:
- 深度学习 (Deep Learning) 是一种深度学习.
- 机器学习 机器学习
- 计算机科学 计算机科学
背景情况:
- 深度学习分类器通常使用SoftMax和CrossEntropy成本函数.
- 软马克斯将模型得分转换为类概率.
- 交叉度测量了从目标分布的预测差异.
研究的目的:
- 为了引入改进的软马克斯前 (ISBE) 功能.
- 为了证明深度学习中交叉计算的冗余性.
- 为了优化深度学习分类器训练和反向传播.
主要方法:
- 引入ISBE的功能. 引入ISBE的功能.
- 在没有交叉计算的情况下对反向传播的分析.
- 使用MNIST数据集对感知子和卷积网络的实施和测试.
- 正式证明软马克斯技巧的适用性.
主要成果:
- ISBE省略了交叉计算,简化了反向传播.
- 仅使用SoftMax来维持分类器的性能,并具有各种激活功能.
- 在前向和后向传票中观察到显著的运营节省.
- ISBE与SoftMax,Sigmoid,Tanh,HardSigmoid和HardTanh一起有效. 这两种方法中的一个是:
结论:
- 对深度学习分类器来说,交叉计算是不必要的.
- ISBE提供了计算节约,并保持了性能.
- 提出的方法适用于程序员和设计深度模型的学生.
相关概念视频
Standard Error of the Mean
5.8K
The sampling variability of a statistic is defined as how much the statistic varies from one sample to another. The sampling variability of a statistic is typically measured by measuring its standard error.
The standard error of the mean is an example of a standard error. It is a unique standard deviation known as the standard deviation of the sampling distribution of the mean. The standard error of the mean is a statistic that calculates how correctly a sample distribution represents a...
The standard error of the mean is an example of a standard error. It is a unique standard deviation known as the standard deviation of the sampling distribution of the mean. The standard error of the mean is a statistic that calculates how correctly a sample distribution represents a...
5.8K
Margin of Error
4.1K
The margin of error is also called the maximum error of an estimate. The margin of error is the maximum possible or expected difference between the observed sample parameter value and the actual population parameter value. For proportion, it is the maximum difference between the value of sample proportion obtained from the data and the true value of population proportion. As the true value of the population parameter is not known, the margin of error is calculated using the sample statistic.
4.1K
Detection of Gross Error: The Q Test
6.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.1K
Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
Types of Errors: Detection and Minimization
1.6K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
1.6K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K


