一个深度神经网络规范化措施:基于类的脱关系方法
Chenguang Zhang1, Tian Liu2, Xuejiao Du1
1School of Mathematics and Statistics, Hainan University, Haikou 570100, China.
Entropy (Basel, Switzerland)
|January 26, 2024
概括
这项研究引入了一种新的规范化方法,即基于类的脱关系方法 (CDM),用于对抗深度学习模型的过拟合. 通过促进神经元多样性和特定类的凝聚力,CDM提高了模型的准确性和概括性.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 人工智能的人工智能
背景情况:
- 过度装配在深度学习中构成了重大挑战,降低了网络通用化和性能.
- 现有的规范化技术可能无法完全解决神经元相关性和分类准确性之间的复杂相互作用.
研究的目的:
- 引入一种新的规范化技术,即基于类的脱相关方法 (CDM).
- 通过解决隐藏层内的神经元相关性来增强网络概括和模型准确性.
主要方法:
- 基于类的脱相关方法 (CDM) 将隐藏层神经元视为基础学习者.
- CDM将基础学习者之间的相关性最小化,同时最大化了类条件相关性.
- 该方法促进神经元之间的多样性和特定类的凝聚力.
主要成果:
- 使用深度模型对各种数据集的实验表明了CDM的有效性.
- 在深度学习网络中,CDM显著减少了过度匹配.
- 通过CDM,分类性能和模型准确性得到了明显的改善.
结论:
- 基于类的脱相关方法 (CDM) 是深度学习的一个有前途的规范化技术.
- CDM提供了促进神经元多样性和增强类特定特征学习的双重好处.
- 这种方法有效地打击过拟合,导致深度模型的优越泛化和准确性.
相关概念视频
Multiple Regression
3.0K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.0K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Residuals and Least-Squares Property
7.4K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.4K
Regression Analysis
5.7K
Regression analysis is a statistical tool that describes a mathematical relationship between a dependent variable and one or more independent variables.
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
5.7K
Reducing Line Loss
154
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
154
Deconvolution
160
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
160


