规范化,早期停止和梦想:一种类似霍普菲尔德的设置,以解决泛化和过度拟合的问题
E Agliari1, F Alemanno2, M Aquaro1
1Dipartimento di Matematica "Guido Castelnuovo", Sapienza Università di Roma, Italy; GNFM-INdAM, Gruppo Nazionale di Fisica Matematica (Istituto Nazionale di Alta Matematica), Italy.
概括
这项研究利用机器学习优化了吸引神经网络,发现 Hebbian 学习与取消学习避免了过拟合. 规范化和早期停止等策略增强了网络泛化能力.
科学领域:
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
背景情况:
- 吸引神经网络 (ANN) 是由生物神经网络启发的计算模型.
- 传统的ANN经常面临过度适应和泛化方面的挑战.
- 机器学习视角为优化ANN参数提供了新的方法.
研究的目的:
- 应用机器学习技术,特别是调节损失函数上的梯度下降,以优化吸引器神经网络参数.
- 调查ANN中去学习协议,规范化和培训时间之间的关系.
- 分析优化ANN在不同数据模式中的概括能力.
主要方法:
- 在正规化损失函数上利用梯度下降优化来确定最佳网络参数.
- 确定了最佳的神经元交互矩阵作为赫比安内核,经由一个失学协议修改.
- 在合成数据集上进行分析和数值实验,以评估网络性能和通用化.
主要成果:
- 不学习协议的范围与规范化超参数和训练持续时间直接相关.
- 通过规范化和早期停止开发了避免过度装配的策略.
- 根据数据集参数确定了不同的性能模式 (过度装配,故障,成功).
结论:
- 由unlearning修改的Hebbian内核提供了一种有效的方法,可以在机器学习框架内训练ANN.
- 规范化和早期停止对于管理过度装配和增强ANN中的泛化至关重要.
- 该研究为设计和分析具有更好的预测性能的ANN提供了一个框架.
相关概念视频
Generalization, Discrimination, and Extinction
528
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
528
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
Residuals and Least-Squares Property
7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K
Propagation of Uncertainty from Random Error
680
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
680
Multiple Regression
3.0K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.0K


