在自主监督学习中理解美白损失
概括
本研究分析了自主监督学习 (SSL) 中的白化损失,以防止功能崩. 一种新的方法,随机组分区 (CW-RGP) 的通道白化,避免了没有大批次大小的崩,显示了对表示学习的希望.
科学领域:
- 计算机科学 计算机科学
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 自主监督学习 (SSL) 旨在学习没有标记数据的表示.
- 特性崩是SSL的一个关键挑战,在SSL中,不同的输入映射到相同的表示.
- 白化损失是一种通过确保白化嵌入来防止特征崩的技术.
研究的目的:
- 分析白化损失及其在防止SSL中的功能崩中的作用.
- 为了消除与白化损失相关的现象,并将其与其他SSL方法连接起来.
- 提出一种超越现有美白技术局限性的新方法.
主要方法:
- 开发了一个分析框架和一个指标来研究美白损失.
- 证明批量美白 (BW) 需要全级嵌入,而不是严格的美白.
- 根据分析,建议使用随机组分区 (CW-RGP) 进行通道白化.
主要成果:
- 证明全等级约束足以避免尺寸崩.
- 在特定条件下,在梯度下降过程中证明了稳定的等级不变性.
- 在不需要大批量尺寸的情况下,CW-RGP有效地防止了崩.
结论:
- 拟议的CW-RGP方法利用了BW的优势,同时减轻了其缺点.
- CW-RGP显示了学习高质量的表示的巨大潜力.
- 分析提供了关于白化损失及其与其他SSL方法的联系的见解.
相关概念视频
Survival Tree
79
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
79
Reducing Line Loss
150
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
150
Self-Evaluation: Self-Enhancement and Self-Verification
5.2K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.2K
Residuals and Least-Squares Property
7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K
Stereotype Threat and Self-fulfilling Prophecies
37.6K
When we hold a stereotype about a person, we have expectations that he or she will fulfill that stereotype. A self-fulfilling prophecy is an expectation held by a person that alters his or her behavior in a way that tends to make it true. When we hold stereotypes about a person, we tend to treat the person according to our expectations. This treatment can influence the person to act according to our stereotypic expectations, thus confirming our stereotypic beliefs. Research by Rosenthal and...
37.6K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K


