在机器学习中交叉损失的相关性
Richard Connor1, Alan Dearle1, Ben Claydon1
1School of Computer Science, University of St Andrews, St Andrews KY16 9SS, UK.
Entropy (Basel, Switzerland)
|June 26, 2024
概括
深度学习中的交叉损失与三角分歧和欧几里德距离对逻辑的强烈相关. 这表明三角分歧作为一个具有成本效益的替代方案,并使用欧几里德距离来验证分类中的逻辑相似性.
科学领域:
- 机器学习 机器学习
- 深度神经网络 深度神经网络
- 信息理论 信息理论
背景情况:
- 交叉损失对于训练深度神经网络至关重要.
- 了解损失函数之间的关系可以优化训练和特征提取.
研究的目的:
- 为了研究交叉和其他差异函数之间的新相关性.
- 为了探索交叉和欧几里德距离在逻辑上之间的关系.
- 识别潜在的计算效率和新的特征表示.
主要方法:
- 实证观察差异函数之间的相关性.
- 数学分析相关的分歧函数和软max输出.
- 对交叉,三角分歧和欧几里德距离对逻辑的分析.
主要成果:
- 在某些场景中证明了交叉和三角分歧之间的近乎完美的相关性.
- 在软max衍生逻辑上显示了交叉和欧几里德距离之间的强烈相关性.
- 确定逻辑可以有效地被视为Euclidean空间中的特征.
结论:
- 三角分歧在计算上提供了一个比交叉损失更便宜的替代方案.
- 欧几里德对逻辑的距离是用软max和交叉训练的网络相似性的有效和协同措施.
- 这些发现为优化深度学习模型培训和分析提供了实际意义.
相关概念视频
Correlation and Regression
1.2K
In statistics, correlation describes the degree of association between two variables. In the subfield of linear regression, correlation is mathematically expressed by the correlation coefficient, which describes the strength and direction of the relationship between two variables. The coefficient is symbolically represented by 'r' and ranges from -1 to +1. A positive value indicates a positive correlation where the two variables move in the same direction. A negative value suggests a...
1.2K
Correlations
32.8K
Correlation means that there is a relationship between two or more variables (such as ice cream consumption and crime), but this relationship does not necessarily imply cause and effect. When two variables are correlated, it simply means that as one variable changes, so does the other. We can measure correlation by calculating a statistic known as a correlation coefficient. A correlation coefficient is a number from -1 to +1 that indicates the strength and direction of the relationship between...
32.8K
Coefficient of Correlation
6.1K
The correlation coefficient, r, developed by Karl Pearson in the early 1900s, is numerical and provides a measure of strength and direction of the linear association between the independent variable x and the dependent variable y.
If you suspect a linear relationship between x and y, then r can measure how strong the linear relationship is.
What the VALUE of r tells us:
The value of r is always between –1 and +1: –1 ≤ r ≤ 1.
The size of the correlation r indicates the...
If you suspect a linear relationship between x and y, then r can measure how strong the linear relationship is.
What the VALUE of r tells us:
The value of r is always between –1 and +1: –1 ≤ r ≤ 1.
The size of the correlation r indicates the...
6.1K
Calibration Curves: Correlation Coefficient
1.6K
In a linear calibration curve, there is a value called the calibration coefficient, denoted by 'r,' which measures the strength and the direction of association between two variables. The correlation coefficient value ranges from −1 to +1. A value of +1 indicates a perfect positive linear correlation, −1 denotes a perfect negative correlation, and 0 implies no correlation between the two variables. A positive correlation value establishes that as one variable increases, the...
1.6K
Correlation
11.7K
In statistics, two variables are said to be correlated if the values of one variable are associated with the other variable. Depending on the relationship between two variables, correlation can be of three types– positive correlation, negative correlation, and zero correlation.
Two variables, for example, a and b, are said to be positively correlated if both variables move in the same direction. In other words, a positive correlation exists between two variables, a and b, if:
Two variables, for example, a and b, are said to be positively correlated if both variables move in the same direction. In other words, a positive correlation exists between two variables, a and b, if:
11.7K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K


