使用修正线性单位估计无监督学习率的分析方法
Chaoxiang Chen1,2,3, Vladimir Golovko4,5, Aliaksandr Kroshchanka5
1School of Information Science and Technology, Zhejiang Shuren University, Hangzhou, China.
Frontiers in neuroscience
|April 23, 2024
概括
这项研究引入了使用ReLU的受限制博尔兹曼机器 (RBM) 的自适应学习速率,自动优化神经网络步骤以获得更好的性能. 该方法的表现优于常量步骤和亚当方法在概括和减少损失方面.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 神经网络的神经网络的神经网络
背景情况:
- 无监督学习,特别是受限制的博尔兹曼机器 (RBM) 和自动编码器,是神经网络研究的一个关键领域.
- 适应性学习速度对于优化神经网络训练效率和性能至关重要.
研究的目的:
- 提出适应式学习步骤计算的数学表达式,用于具有ReLU转移函数的RBM.
- 为了自动估计和更新学习步骤大小,以最大限度地减少神经网络的损失功能.
主要方法:
- 在RBM中开发用于自适应学习步骤计算的新数学表达式.
- 使用最的下降方法来理论证明适应性学习速率方法.
- 将拟议的自适应方法与常量步骤和亚当方法进行比较.
主要成果:
- 拟议的自适应学习速率方法自动估计的步骤大小,尽量减少损失函数.
- 该技术在每次代中都成功地更新了学习步骤.
- 与现有的常量步和亚当方法相比,在概括能力和损失函数方面表现出卓越的性能.
结论:
- 为带有ReLU的RBM开发的自适应学习率估计技术提供了更好的性能.
- 这种方法提供了一种强大的方法来优化学习步骤大小,增强神经网络训练.
- 这些发现表明,使用RBM来进行无监督学习的方法更有效和更有效.
相关概念视频
Calibration Curves: Linear Least Squares
1.3K
A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
For data that follow a straight line, the standard method for fitting is the linear...
1.3K
Linear Approximation in Frequency Domain
89
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
89
Residuals and Least-Squares Property
7.4K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.4K
Variation
6.8K
An important characteristic of any set of data is the variation in the data. In some data sets, the data values are concentrated closely near the mean; in other data sets, the data values are more widely spread out from the mean. The most common measure of variation, or spread, is the standard deviation, which is the square root of variance.
When independent and dependent variables are plotted on a scatter plot, the slope of a line is a value that describes the rate of change between the two...
When independent and dependent variables are plotted on a scatter plot, the slope of a line is a value that describes the rate of change between the two...
6.8K
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
482
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
482
First Order Systems
89
First-order systems, such as RC circuits, are foundational in understanding dynamic systems due to their straightforward input-output relationship. Analyzing their responses to different input functions under zero initial conditions reveals significant insights into system behavior.
When a first-order system is subjected to a unit-step input, its response is characterized by its transfer function. By applying the Laplace transform of the unit-step input to the transfer function, expanding the...
When a first-order system is subjected to a unit-step input, its response is characterized by its transfer function. By applying the Laplace transform of the unit-step input to the transfer function, expanding the...
89


