球形负感知子的解决方案的星形空间
Brandon Livio Annesi1, Clarissa Lauditi2, Carlo Lucibello1,3
1Department of Computing Sciences, Bocconi University, 20136 Milano, Italy.
Physical review letters
|December 15, 2023
概括
神经网络的能量景观具有复杂的结构. 这项研究揭示了过度参数化的球形感知子具有简单的星形溶液连接性,但在更高密度时会分解.
科学领域:
- 计算神经科学是一种计算神经科学.
- 机器学习理论机器学习理论
- 统计物理学的统计物理.
背景情况:
- 经验研究表明神经网络的低能量的状态形成复杂的连接结构.
- 零能量路径可以连接这些景观中的遥远解决方案.
- 球形负感知子是非凸神经网络的模型.
研究的目的:
- 分析球形负感知子的能量格局和连接性.
- 开发一种用于计算能源障碍的分析方法.
- 了解在过度参数化的模式下解决方案的多元性质.
主要方法:
- 将球形负感知子作为一个连续约束满足问题.
- 开发一种用于计算能源障碍的一般分析方法.
- 从平衡状态采样顶点配置.
- 分析解决方案多元体内的地测连接性.
主要成果:
- 在过度参数化的模式下,解决方案多元体表现出简单的连接性.
- 一个大,地形凸的组件吸引了各种优化动态.
- 高边际解决方案的子集形成星形几何形状,与大多数其他解决方案相连接.
- 连接的解决方案空间的分析特征.
结论:
- 过度参数化的球形负感知子显示了一个简单的,星形的解决方案几何.
- 这种简单的连接方便了优化动态.
- 在较高的约束密度下观察到过渡,在那里地质连接性会崩.
相关概念视频
Spherical Coordinates
10.2K
Spherical coordinate systems are preferred over Cartesian, polar, or cylindrical coordinates for systems with spherical symmetry. For example, to describe the surface of a sphere, Cartesian coordinates require all three coordinates. On the other hand, the spherical coordinate system requires only one parameter: the sphere's radius. As a result, the complicated mathematical calculations become simple. Spherical coordinates are used in science and engineering applications like electric and...
10.2K
Gauss's Law: Spherical Symmetry
7.5K
A charge distribution has spherical symmetry if the density of charge depends only on the distance from a point in space and not on the direction. In other words, if the system is rotated, it doesn't look different. For instance, if a sphere of radius R is uniformly charged with charge density ρ0, then the distribution has spherical symmetry. On the other hand, if a sphere of radius R is charged so that the top half of the sphere has a uniform charge density ρ1 and the bottom half...
7.5K
Root Loci for Positive-Feedback Systems
124
The Hartley oscillator is a positive feedback system that sustains oscillations by feeding the output back to the input in phase, thereby reinforcing the signal. Positive feedback systems can be viewed as negative feedback systems with inverted feedback signals. In these systems, the root locus encompasses all points on the s-plane where the angle of the system transfer function equals 360 degrees.
The construction rules for the root locus in positive feedback systems are similar to those in...
The construction rules for the root locus in positive feedback systems are similar to those in...
124
Polar and Cylindrical Coordinates
14.6K
The Cartesian coordinate system is a very convenient tool to use when describing the displacements and velocities of objects and the forces acting on them. However, it becomes cumbersome when we need to describe the rotation of objects. So, when describing rotation, the polar coordinate system is generally used.
14.6K
Gauss's Law: Problem-Solving
1.7K
Gauss's law helps determine electric fields even though the law is not directly about electric fields but electric flux. In situations with certain symmetries (spherical, cylindrical, or planar) in the charge distribution, the electric field can be deduced based on the knowledge of the electric flux. In these systems, we can find a Gaussian surface S over which the electric field has a constant magnitude. Furthermore, suppose the electric field is parallel (or antiparallel) to the area...
1.7K
Dot Product: Problem Solving
381
The dot product is a powerful tool in problem-solving involving vectors, given that the dot product of two vectors is the product of their magnitudes and the cosine of the angle between them measured anti-clockwise. Solving problems involving the dot product requires understanding its properties and developing a step-by-step process to solve them. Here are the main steps to follow when solving any general problem involving the dot product:
Identify the problem: Start by reading the problem and...
Identify the problem: Start by reading the problem and...
381


