在普遍标签下学习学生网络 噪音
概括
这项研究引入了一种新的无数据知识蒸方法,用于训练使用具有通用标签噪声的网络数据的小型网络. 它有效地处理封闭式和开放式标签噪声,以提高性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 无数据知识蒸 (DFKD) 培养了没有原始数据的学生网络.
- 现有的DFKD方法在网络收集的数据中经常忽略开放式标签噪声.
- 这种限制阻碍了学生网络在现实场景中的表现.
研究的目的:
- 提出一个新的DFKD范式,解决通用标签噪声 (封闭式和开放式).
- 通过有效利用网络收集的数据与混合标签噪音来增强学生的网络学习.
- 为了提高蒸模型的强度和性能.
主要方法:
- 收集的网络数据与通用标签噪声.
- 根据预测不确定性,将数据划分为干净,封闭噪声和开放噪声集.
- 使用教师网络,为封闭噪声数据精细化标签.
- 在干净和精致的闭合噪音设置上采用噪音强的混合对比学习.
- 使用自主监督学习对开放噪音 (未标记) 数据.
主要成果:
- 拟议的方法在图像分类任务上明显优于现有的DFKD方法.
- 在处理封闭式和开放式标签噪声方面表现出卓越的性能.
- 从教师到学生网络实现了有效的知识传输,即使在有噪音数据的情况下.
结论:
- 新的DFKD范式有效地解决了通用标签噪声,包括以前被忽视的开放式噪声.
- 标签改进,对比学习和自我监督学习的混合方法提高了蒸.
- 这项工作为使用网络数据进行无数据知识提炼提供了更现实的和有效的解决方案.
相关概念视频
01:31Student t Distribution
6.0K
The population standard deviation is rarely known in many day-to-day examples of statistics. When the sample sizes are large, it is easy to estimate the population standard deviation using a confidence interval, which provides results close enough to the original value. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
The Student t distribution was developed by William S. Goset (1876–1937) of the...
The Student t distribution was developed by William S. Goset (1876–1937) of the...
6.0K
01:25Network Function of a Circuit
276
Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.
276
00:59Propagation of Uncertainty from Random Error
661
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
661
01:19Uniform Distribution
4.8K
The uniform distribution is a continuous probability distribution of events with an equal probability of occurrence. This distribution is rectangular.
Two essential properties of this distribution are
Two essential properties of this distribution are
4.8K
01:10Propagation of Uncertainty from Systematic Error
496
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
496
01:18Introduction to Learning
357
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
357