对于回归问题的多尺度神经网络的光谱偏差减少
概括
本研究引入了扩散方程模型来分析多尺度深度神经网络 (MscaleDNN) 的训练错误. 这些模型解释了 MscaleDNN 如何减少光谱偏差,改善跨不同频率的函数近似度.
科学领域:
- 机器学习 机器学习
- 深度学习理论 深度学习理论
- 数字分析 数字分析
背景情况:
- 深度神经网络表现出光谱偏差,限制了它们近似振荡函数的能力.
- 多尺度深度神经网络 (MscaleDNN) 旨在减轻这种光谱偏差.
研究的目的:
- 在光谱领域推导和验证扩散方程模型,用于分析 MscaleDNN 训练错误.
- 解释 MscaleDNN 减少光谱偏差的机制.
主要方法:
- 从 MscaleDNN 的光谱误差方程中导出扩散模型,使用神经触角内核.
- 假设包括梯度下降训练,正弦激活,消失学习速率和无限网络宽度/域大小.
- 福里埃光谱域的分析,以了解跨频率的训练误差衰变.
主要成果:
- 当在 MscaleDNN 中使用更多尺度时,观察到具有更大的支的扩散系数.
- 衍生出的扩散模型准确地预测了实际MscaleDNN训练的错误演变.
- MscaleDNN在广泛的频率范围内展示了错误衰减,证实了它的优势.
结论:
- 拟议的扩散方程模型有效地解释了MscaleDNN的光谱偏差减少能力.
- 与标准DNN相比,MscaleDNN提供了与不同频率组件相近功能的卓越性能.
相关概念视频
Multiple Regression
2.9K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
2.9K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Calibration Curves: Linear Least Squares
1.2K
A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
For data that follow a straight line, the standard method for fitting is the linear...
1.2K
Multi-input and Multi-variable systems
94
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
94
Residuals and Least-Squares Property
7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K


