镜子下降和指数梯度算法使用痕迹形式度
Andrzej Cichocki1,2,3,4, Toshihisa Tanaka3, Frank Nielsen5
1Systems Research Institute of Polish Academy of Science, Newelska 6, 01-447 Warsaw, Poland.
Entropy (Basel, Switzerland)
|December 24, 2025
概括
这项研究提出了新的镜像下降 (MD) 和使用通用的通用指数梯度 (GEG) 算法. 这些方法通过适应复杂的几何形状,提供了更好的收性和稳定性.
科学领域:
- 优化理论 优化理论
- 信息几何学信息几何学
- 机器学习 机器学习
背景情况:
- 镜像下降 (MD) 和通用指数梯度 (GEG) 是基本的优化算法.
- 经典方法经常与消失/爆炸梯度和非欧几里德几何学作斗争.
- 一般化的率为定义分歧和指标提供了一个灵活的框架.
研究的目的:
- 引入MD和GEG算法的统一框架,基于概括的痕迹形式.
- 为了证明这些新算法的改进的收性和稳定性特性.
- 揭示这些方法与自然梯度下降相连接的信息几何基础.
主要方法:
- 通过变形的对数来导出MD和GEG算法从痕迹形式的值.
- 对收行为和梯度强度的分析.
- 调查与阿玛里的自然梯度和信息几何结构的联系.
- 适用于特定的家族 (Tsallis,Kaniadakis等) 为了定义里曼的指标.
主要成果:
- 开发一类广泛的MD和GEG算法,以增强的融合和稳定性.
- 为各种梯度更新规则 (附加,乘法,自然) 建立统一的几何基础.
- 证明不同的值诱导不同的里曼度量,保持统计几何学.
- 可调节的参数允许自适应的几何选择,以提高优化.
结论:
- 拟议的框架将第一阶段优化方法统一在一般化的布雷格曼分歧下.
- 的选择决定了底层的里曼度量和双重几何结构.
- 与经典欧几里德优化相比,这些通用方法提供了增强的适应性和稳定性.
相关概念视频
Gradient and Del Operator
4.3K
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
4.3K
Second Derivatives and Laplace Operator
2.6K
The first order operators using the del operator include the gradient, divergence and curl. Certain combinations of first order operators on a scalar or vector function yield second order expressions. Second-order expressions play a very important role in mathematics and physics. Some second order expressions include the divergence and curl of a gradient function, the divergence and curl of a curl function, and the gradient of a divergence function.
Consider a scalar function. The curl of its...
Consider a scalar function. The curl of its...
2.6K
Divergence and Stokes' Theorems
3.4K
The divergence and Stokes' theorems are a variation of Green's theorem in a higher dimension. They are also a generalization of the fundamental theorem of calculus. The divergence theorem and Stokes' theorem are in a way similar to each other; The divergence theorem relates to the dot product of a vector, while Stokes' theorem relates to the curl of a vector. Many applications in physics and engineering make use of the divergence and Stokes' theorems, enabling us to write...
3.4K
Entropy Change in Reversible Processes
3.2K
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
3.2K
Methods of Obtaining Topography
257
Topography involves measuring and mapping land elevations, natural features, and artificial structures to create accurate representations of the terrain. Topographic surveying relies on traditional and modern methods, each with distinct advantages and limitations.Traditional Surveying Methods:Transit stadia surveys and plane table surveys were widely used traditional surveying methods. These techniques relied on instruments like theodolites and stadia rods for measuring distances and angles,...
257
Forced Transdifferentiation
2.3K
Transdifferentiation, also known as lineage reprogramming, was first discovered by Selman and Kafatos in 1974 in silkmoths. They observed that the moths’ cuticle-producing cells transformed into salt-producing cells. Many such cases of natural transdifferentiation occur in organisms. In humans, pancreatic alpha cells can become beta cells. In newts, the loss of the eye’s lens causes the pigmented epithelial cells to transdifferentiate into the lens cells.
Artificial...
Artificial...
2.3K


