深度表示学习使用层级智能的VICReg损失
Joy Datta1, Rawhatur Rabbi1, Puja Saha2
1Department of Computer Science and Engineering, School of Data and Sciences, Brac University, Dhaka, Bangladesh.
Scientific reports
|July 27, 2025
概括
本研究介绍了一种使用变量-不变性-共变性调节 (VICReg) 损失的层级神经网络训练方法. 这种方法通过创建紧而有信息的特征表示来提高分类准确性,特别是在有限的标记数据中.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 计算机视觉 计算机视觉
背景情况:
- 深度神经网络 (DNN) 经常与消失梯度和初始化灵敏度作斗争.
- 训练DNN通常需要大量的注释数据,这些数据可能很少.
- 背向传播,标准的训练方法,涉及一个向前和一个向后传球.
研究的目的:
- 为神经网络提供一种新的层级智能训练程序.
- 为了应对培训DNN的挑战,特别是有限的注释数据.
- 为了增强特征表示学习和分类准确性.
主要方法:
- 一个层级明智的训练程序,最大限度地减少每个层的变异-不变-共变调整 (VICReg) 损失.
- 使用两个前向传递与原始和增强数据,而不是反向传播.
- 采用金字塔式网络架构来有效地提取特征.
- 优化对差异,不变性和共变性术语的权重,以获取语义信息.
主要成果:
- 该程序逐步构建紧和信息化的特征空间.
- 与基线模型相比,MNIST (7%),EMNIST (16%),时尚MNIST (1%) 和CIFAR-100 (7%) 的分类准确性得到改善.
- 通过集群质量指标和少数镜头分类任务来评估学习的表示.
结论:
- 拟议的VICReg层级培训提高了DNN的性能,特别是在低数据的系统中.
- 这种方法为反向传播提供了一个可行的替代方案,减轻了常见的培训问题.
- 这种方法有效地学习了下游任务的强大和有信息的表示形式.
相关概念视频
01:18Reducing Line Loss
193
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
193
01:27Associative Learning
566
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
566
01:11Residuals and Least-Squares Property
7.8K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.8K
01:37Neural Regulation
39.8K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.8K
01:18Introduction to Learning
528
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
528
01:25Multiple Regression
3.2K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.2K