有随机标签的感知子的高斯普遍性
Federica Gerace1,2, Florent Krzakala3, Bruno Loureiro3,4
1International School of Advanced Studies (SISSA), Trieste, Via Bonomea, 265, 34136 Trieste, Italy.
Physical review. E
|April 18, 2024
概括
这项研究表明,随机标签的通用线性分类在许多高维数据类型中实现了与高斯数据相同的最小训练损失. 这种训练损失变得独立于数据共变性,随着规范化消失.
科学领域:
- 机器学习 机器学习
- 统计物理 统计物理
- 数据科学数据科学数据科学
背景情况:
- 假设高斯式,独立且相同分布的 (i.i.d.) 输入数据是统计和机器学习研究中常见的局限性.
- 在实际应用中,这种假设通常被认为是限制性的.
研究的目的:
- 重新评估高斯式i.i.d.的意义 在通用线性分类 (感知子模型) 中使用随机标签的数据假设.
- 展示高维输入数据的普遍性类别,其性能与高斯数据相当.
主要方法:
- 用随机标签对通用线性分类进行理论分析.
- 证明同质高斯云的任意混合的普遍性.
- 在不同的现实世界数据集上进行经验验证.
主要成果:
- 高维输入数据的大型通用性类可以实现与高斯数据相同的最小训练损失,并且具有匹配的共变量.
- 在消失规范化的极限中,训练损失变得独立于数据共变量.
- 对高斯混合模型的普遍性的理论证明.
结论:
- 这些发现扩大了理论模型的适用性,超出了严格的高斯式i.i.d. 一个假设,一个假设.
- 随机标签的感知子模型的普遍性在理论上和经验上都得到了证明.
- 这项研究减轻了机器学习中高斯数据假设的感知局限性.
相关概念视频
Gauss's Law
7.3K
If a closed surface does not have any charge inside where an electric field line can terminate, then the electric field line entering the surface at one point must necessarily exit at some other point of the surface. Therefore, if a closed surface does not have any charges inside the enclosed volume, then the electric flux through the surface is zero. What happens to the electric flux if there are some charges inside the enclosed volume? Gauss's law gives a quantitative answer to this question.
7.3K
Random Variables
11.8K
A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
11.8K
Gauss's Law in Dielectrics
4.4K
Consider a polar dielectric placed in an external field. In such a dielectric, opposite charges on adjacent dipoles neutralize each other, such that the net charge within the dielectric is zero. When a polar dielectric is inserted in between the capacitor plates, an electric field is generated due to the presence of net charges near the edge of the dielectric and the metal plates interface. Since the external electrical field merely aligns the dipoles, the dielectric as a whole is neutral. An...
4.4K
Probability Distributions
6.9K
The probability of a random variable x is the likelihood of its occurrence. A probability distribution represents the probabilities of a random variable using a formula, graph, or table. There are two types of probability distribution– discrete probability distribution and continuous probability distribution.
A discrete probability distribution is a probability distribution of discrete random variables. It can be categorized into binomial probability distribution and Poisson...
A discrete probability distribution is a probability distribution of discrete random variables. It can be categorized into binomial probability distribution and Poisson...
6.9K
Gauss's Law: Planar Symmetry
7.9K
A planar symmetry of charge density is obtained when charges are uniformly spread over a large flat surface. In planar symmetry, all points in a plane parallel to the plane of charge are identical with respect to the charges. Suppose the plane of the charge distribution is the xy-plane, and the electric field at a space point P with coordinates (x, y, z) is to be determined. Since the charge density is the same at all (x, y) - coordinates in the z = 0 plane, by symmetry, the electric field at P...
7.9K
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106


