适应性随机镜子下降的收
IEEE transactions on neural networks and learning systems
|March 18, 2025
概括
本研究介绍了使用镜像图的自适应性随机优化方法,解释了自适应时刻估计 (Adam) 类型算法的成功. 这些方法为凸函数和强烈凸函数提供了更好的收率.
科学领域:
- 优化理论 优化理论
- 机器学习算法 机器学习算法
- 凸的分析 凸的分析
背景情况:
- 适应性随机优化方法对于机器学习至关重要.
- 镜像地图在优化中捕获几何属性.
- 适应时刻估计 (Adam) 类型的算法被广泛使用,但缺乏超参数选择的理论依据.
研究的目的:
- 介绍一系列基于镜像图的适应性随机优化方法.
- 从理论上分析这些方法对各种函数类型的收率.
- 为亚当型算法中超参数选择提供解释.
主要方法:
- 开发了结合镜像图的自适应性随机优化算法.
- 分析凸和强烈凸的目标函数的平均遗憾收率.
- 使用强的可微分镜像图的属性,研究光滑,非凸函数的收性.
主要成果:
- 在标准假设下实现凸目标函数的收率.
- 强烈凸起的目标函数的更好的收率.
- 证明了顺序的合率,直至顺的客观函数的对数项,与实际的亚当类型算法使用保持一致.
结论:
- 拟议的适应性随机优化方法家族提供了理论上的保证.
- 该研究提供了对亚当类型算法的有效性和超参数选择的见解.
- 这项工作弥合了理论分析和适应性优化的实际应用之间的差距.
更多相关视频
06:45Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
Published on: October 28, 2022
1.5K
07:05Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
Published on: October 27, 2016
9.2K
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
37
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
37
Region of Convergence
347
The z-transform is a powerful mathematical tool used in the analysis of discrete-time signals and systems. It is a crucial tool in the analysis of discrete-time systems, but its convergence is limited to specific values of the complex variable z. This range of values, known as the Region of Convergence (ROC), is fundamental in determining the behavior and stability of a system or signal. The ROC defines the region in the complex plane where the z-transform converges, which can take various...
347
Region of Convergence of Laplace Tarnsform
452
The Region of Convergence (ROC) is a fundamental concept in signal processing and system analysis, particularly associated with the Laplace transform. The ROC represents an area in the complex plane where the Laplace transform of a given signal converges, determining the transform's applicability and utility.
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This...
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This...
452
