相关实验视频
Updated: May 27, 2025

06:37
Quantifying Cytoskeleton Dynamics Using Differential Dynamic Microscopy
Published on: June 15, 2022
3.4K
在学习和戴森-布朗运动期间的随机权重矩阵动态
Gert Aarts1, Biagio Lucini2, Chanju Park1
1Swansea University, Department of Physics, Swansea SA2 8PP, United Kingdom.
Physical review. E
|February 20, 2025
概括
我们表明,学习算法的权重矩阵更新遵循戴森-布朗运动,将随机性与学习速率和迷你批量大小联系起来. 这揭示了机器学习模型中随机矩阵理论的普遍特征.
科学领域:
- 机器学习 机器学习
- 统计物理 统计物理
- 随机矩阵理论 随机矩阵理论
背景情况:
- 重量矩阵更新是机器学习算法的核心.
- 了解这些更新的动态对于算法性能和通用化至关重要.
- 现有的理论往往缺乏一个统一的框架来解释这些更新的随机性质.
研究的目的:
- 为了证明重量矩阵更新可以使用戴森-布朗运动来建模.
- 建立学习的随机性和学习率与小批量大小的比率之间的联系.
- 在得到的分布中识别通用和非通用特征,并将它们与已建立的随机矩阵理论概念联系起来.
主要方法:
- 应用戴森-布朗运动框架来分析重量矩阵更新.
- 与学习率和迷你批量大小比率与随机性水平的关系.
- 分析库伦气体分布的固有值.
- 在模型中识别特定的随机矩阵理论分布 (维格纳推测,半圆).
主要成果:
- 重量矩阵更新被证明可以通过戴森-布朗运动来描述.
- 证实了随机性和学习率/小批量大小比率之间的缩放关系.
- 讨论了库伦气体分布的通用和非通用特征.
- 维格纳推测和半圆分布在老师-学生和高斯限制的博尔兹曼机器模型中被明确识别出来.
结论:
- 戴森-布朗运动为理解学习算法动态提供了一个强大的框架.
- 这些发现为机器学习中推测的扩展关系提供了强有力的证据.
- 这项工作将随机矩阵理论和机器学习的概念结合起来,为模型行为提供了新的见解.
相关概念视频
Linear Approximation in Time Domain
59
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
59
Damped Oscillations
5.6K
In the real world, oscillations seldom follow true simple harmonic motion. A system that continues its motion indefinitely without losing its amplitude is termed undamped. However, friction of some sort usually dampens the motion, so it fades away or needs more force to continue. For example, a guitar string stops oscillating a few seconds after being plucked. Similarly, one must continually push a swing to keep a child swinging on a playground.
Although friction and other non-conservative...
Although friction and other non-conservative...
5.6K
First Law: Particles in Two-dimensional Equilibrium
5.0K
Recall that a particle in equilibrium is one for which the external forces are balanced. Static equilibrium involves objects at rest, and dynamic equilibrium involves objects in motion without acceleration; but it is important to remember that these conditions are relative. For instance, an object may be at rest when viewed from one frame of reference, but that same object would appear to be in motion when viewed by someone moving at a constant velocity.
Newton's first law tells us about...
Newton's first law tells us about...
5.0K
First Law: Particles in One-dimensional Equilibrium
6.8K
Newton's first law of motion states that a body at rest remains at rest, or if in motion, remains in motion at constant velocity, unless acted on by a net external force. It also states that there must be a cause for any change in velocity (a change in either magnitude or direction) to occur. This cause is a net external force. For example, consider what happens to an object sliding along a rough horizontal surface. The object quickly grinds to a halt, due to the net force of friction. If...
6.8K
Maxwell-Boltzmann Distribution: Problem Solving
1.4K
Individual molecules in a gas move in random directions, but a gas containing numerous molecules has a predictable distribution of molecular speeds, which is known as the Maxwell-Boltzmann distribution, f(v).
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
1.4K
Stability of Equilibrium Configuration
424
Understanding the stability of equilibrium configurations is a fundamental part of mechanical engineering. In any system, there are three distinct types of equilibrium: stable, neutral, and unstable.
A stable equilibrium occurs when a system tends to return to its original position when given a small displacement, and the potential energy is at its minimum. An example of a stable equilibrium is when a cantilever beam is fixed at one end and a weight is attached to the other end. If the weight...
A stable equilibrium occurs when a system tends to return to its original position when given a small displacement, and the potential energy is at its minimum. An example of a stable equilibrium is when a cantilever beam is fixed at one end and a weight is attached to the other end. If the weight...
424

