SSR1M:一种随机的SR1方法,具有动量加速,用于非凸的优化
Jinlan Liu1, Hanger Liu2, Xin Deng2
1Department of Mathematics, Changchun Normal University, Changchun, 130032, China.
概括
一个新的动量 (SSR1M) 的随机对称排列一 (SR1) 算法为大型机器学习问题提供了高效的优化. 这种方法降低了计算成本,并提高了非凸设置中的性能.
科学领域:
- 机器学习 机器学习
- 优化算法 优化算法
- 计算科学 计算科学
背景情况:
- 基于矩阵的二级优化方法表现出强的性能,但由于高计算和存储成本而面临可扩展性挑战.
- 大规模的机器学习问题需要更高效的优化技术,以平衡性能与资源限制.
研究的目的:
- 引入一种新的优化算法,即随机对称排名第一的动量 (SSR1M),旨在克服传统矩阵式方法的局限性.
- 通过向量级操作和内置的技术,如动量和平均值来提高二次优化的实用性和性能.
主要方法:
- 通过用随机梯度取代批量梯度,并将校正项简化为超参数来开发SSR1M.
- 集成的动量,指数移动平均和正常化技术来提高算法性能.
- 在非凸的环境中对SSR1M进行了详细的融合分析.
主要成果:
- 通过使用矢量级操作而不是矩阵产品,SSR1M显著降低了存储和计算开销.
- 对基准非凸型机器学习任务的实证评估表明SSR1M优于最先进的方法.
- 在函数值和优化非凸多项回归和深度神经网络的准确性方面表现出卓越的性能.
结论:
- SSR1M为优化复杂的非形机器学习模型提供了一种高效和实用的解决方案.
- 该算法的设计使其适用于传统方法在计算上不可行的大规模应用.
- 该研究验证了SSR1M的有效性,并提供了一个开源实现,以便更广泛地采用.
相关概念视频
Linear Momentum in Control Volume
1.3K
Newton's second law is applied to obtain the linear momentum in a control volume in a fluid system. According to this law, the rate of change of linear momentum is equal to the sum of external forces acting on the system. When a control volume matches the fluid system at a specific moment, the forces acting on both are identical. Reynolds transport theorem helps explain this by breaking down the system's linear momentum into two components: the rate of change of linear momentum within...
1.3K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
271
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
271
Application of the Linear Momentum Equation
398
The application of the linear momentum equation can be used to analyze the forces needed to hold a 180-degree pipe bend in place with flowing water. In this case, water flows through the bend with a constant cross-sectional area of 0.01 square meters and a flow velocity of 15 meters per second. The pressure at the entrance is 0.2 Megapascals and the pressure at the exit is 0.16 Megapascals.
The goal is to determine the force components in the x and y directions to hold the pipe in place. Since...
The goal is to determine the force components in the x and y directions to hold the pipe in place. Since...
398
Principle of Linear Impulse and Momentum for a Single Particle: Problem Solving
972
Consider a wooden box and a cylinder of known masses m1 and m2, respectively, hanging from a ceiling with the help of a massless pulley system.
972
Linear Momentum
17.5K
The term momentum is used in various ways in everyday language, most of which are consistent with the precise scientific definition. Generally, momentum implies a tendency to continue on course—to move in the same direction; we tend to speak of sports teams or politicians gaining and maintaining the momentum to win. Momentum is also associated with great mass and speed and is often considered when talking about collisions. For example, when rugby players collide and fall to the...
17.5K
Principle of Linear Impulse and Momentum for a Single Particle
1.5K
Linear momentum is a fundamental concept in physics that describes the motion of an object. It is a vector quantity, having a magnitude equal to the product of its mass and its velocity, and direction along the object's velocity. On the other hand, linear impulse, also known as momentum impulse, is a concept in physics related to the change in the linear momentum of an object. Impulse is a vector quantity defined as the product of force and the time over which the force is applied.
Delving...
Delving...
1.5K
