GenEFT:通过物理启发的有效理论理解模型概括的静态和动态
David D Baek1, Ziming Liu1, Max Tegmark1
1Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA.
Physical review. E
|April 18, 2025
概括
我们介绍GenEFT,这是理解神经网络概括的新框架. 它使用物理启发的有效理论来解释概括和表示学习动态,在机器学习中架起理论和实践的桥梁.
科学领域:
- 机器学习 机器学习
- 理论物理 理论物理
- 图表学习学习图表学习
背景情况:
- 神经网络的概括对于模型的性能至关重要.
- 了解概括和超拟合之间的过渡仍然是一个挑战.
- 现有的理论框架往往难以与实际应用相连.
研究的目的:
- 介绍GenEFT,一个有效的理论框架来分析神经网络概括.
- 为了研究随着数据大小的增加而发生的概括阶段过渡.
- 使用相互作用的粒子来建模表示学习动态.
主要方法:
- 开发了GenEFT (一般化有效场理论) 框架.
- 应用GenEFT来图形学习问题.
- 模拟潜伏空间表示作为相互作用的粒子 ("反响").
- 通过扫描编码器和解码器学习率来分析相位过渡.
主要成果:
- 基因EFT有效地解释了神经网络泛化的静态和动态.
- 实验结果与基于信息理论的概括阶段过渡的近似结果保持一致.
- 对表示学习动态的有效理论成功地解释了观察到的概括-过度拟合阶段过渡.
结论:
- GenEFT为理解机器学习提供了一种强大的以物理为灵感的方法.
- 该框架弥合了理论预测和实际机器学习场景之间的差距.
- 有效的理论为推进神经网络概括研究提供了一个有希望的方向.
相关概念视频
Models, Theories, and Laws
5.0K
Scientists frequently use models to help them comprehend a specific collection of phenomena. In physics, a model is a condensed version of a physical system that is too complex to study thoroughly. One such example is the light wave model; unlike water waves, light waves are typically invisible to us. Nonetheless, it is helpful to think of light as being composed of waves, since investigations show that light behaves like water waves. Since it is impossible to visually see what is genuinely...
5.0K
Linear Approximation in Time Domain
56
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
56
Kinematic Equations: Problem Solving
11.7K
When analyzing one-dimensional motion with constant acceleration, the problem-solving strategy involves identifying the known quantities and choosing the appropriate kinematic equations to solve for the unknowns. Either one or two kinematic equations are needed to solve for the unknowns, depending on the known and unknown quantities. Generally, the number of equations required is the same as the number of unknown quantities in the given example. Two-body pursuit problems always require two...
11.7K
An Introduction to Mechanics
1.7K
Humans have been making ships, shelters, pyramids, weapons, agricultural equipment, and many more items without recording the process or theory behind them for centuries. It would be challenging to document the evolution of mechanics from its origin to the present.
According to records, the history of mechanics starts with Aristotle (384–322 BC). He related mechanics to physical theory, aiming for a universal synthesis.
Newton defined mechanics as the branch of physical science that...
According to records, the history of mechanics starts with Aristotle (384–322 BC). He related mechanics to physical theory, aiming for a universal synthesis.
Newton defined mechanics as the branch of physical science that...
1.7K
Principle of Linear Impulse and Momentum for a System of Particles
222
In the context of a system of particles moving relative to an inertial frame of reference, the equation of motion is a crucial tool for understanding the dynamics of the system. This equation, which accounts for external forces acting on each particle, plays a fundamental role in describing the system's behavior.
Notably, internal forces between particles, occurring in equal and opposite collinear pairs, cancel out and are not part of the equation of motion. This exclusion simplifies the...
Notably, internal forces between particles, occurring in equal and opposite collinear pairs, cancel out and are not part of the equation of motion. This exclusion simplifies the...
222
Static Equilibrium - I
11.7K
A rigid body is said to be in dynamic equilibrium when both its linear and angular acceleration are zero, relative to an inertial frame of reference. This means that a body in equilibrium can be moving, but only when its linear and angular velocities are constant. A rigid body is said to be in static equilibrium when it is at rest in the selected frame of reference. The distinction between static equilibrium (e.g., a state of rest) and dynamic equilibrium (e.g, a state of uniform motion) is...
11.7K


