BDS-Adam优化器集成自适应差异纠正与半自适应梯度光滑
Yichuan Shao1, Shiqian Weng1, Haijing Sun1
1School of Intelligent Science and Information Engineering, Shenyang University, Shenyang, 110044, China.
Scientific reports
|October 22, 2025
概括
一个新的优化器,BDS-Adam,通过修复偏差梯度和提高稳定性来增强深度学习训练. 这种自适应方法可以提高不同数据集的融合速度和模型准确性.
科学领域:
- 机器学习 机器学习
- 深度学习优化优化
- 基于梯度的优化方法
背景情况:
- 亚当优化器存在偏差梯度估计和早期训练不稳定性的问题.
- 这些局限性阻碍了深度学习模型的最佳性能.
研究的目的:
- 介绍BDS-Adam,一个增强的亚当优化器变体.
- 解决偏差梯度并改善深度学习的训练稳定性.
主要方法:
- 实现了一种双路径框架,具有非线性梯度映射和半适应式梯度平滑.
- 利用梯度融合和自适应的二次矩度校正.
- 在非凸的设置下进行了收分析.
主要成果:
- BDS-Adam表现出更好的强度和长期稳定性.
- 与亚当相比,测试准确度提高了9.27% (CIFAR-10),0.08% (MNIST) 和3.00% (胃病理学数据集).
- 经验评估证实了增强的融合速度和概括性.
结论:
- BDS-Adam有效地克服了亚当的局限性.
- 拟议的优化器增强了各种任务的融合和泛化.
- BDS-Adam提供了一个更稳定,更准确的训练选择.
相关概念视频
Calibration Curves: Linear Least Squares
4.1K
A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
For data that follow a straight line, the standard method for fitting is the linear...
4.1K
Differential Leveling
656
Differential leveling is a precise method in surveying used to determine the elevation difference between two points. Its primary goal is to establish accurate vertical measurements to create level surfaces or grade lines critical for designing and constructing infrastructures such as roads, bridges, and buildings.The procedure for differential leveling begins with setting up and leveling the instrument at a point where the benchmark can be seen. The level rod is held on the benchmark (BM), and...
656
Residuals and Least-Squares Property
9.1K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
9.1K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
284
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
284
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
1.1K
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
1.1K
Linearization and Approximation
3
Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...
3


