基于瓦瑟斯坦距离的稳定和快速的深度相互信息最大化
Xing He1,2, Changgen Peng3, Lin Wang2
1State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China.
Entropy (Basel, Switzerland)
|December 23, 2023
概括
我们介绍了瓦瑟斯坦基于远程的深度InfoMax (WDIM),以稳定无监督学习. 在不牺牲分类准确性的情况下,WDIM提高了深度表示学习的培训稳定性和融合速度.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 深度学习优越,但需要大量的标记数据,阻碍了应用程序.
- 无监督学习对于在没有标记数据的情况下推进AI至关重要.
- 深度InfoMax (DIM) 学习深度表示,但由于对抗方法而遭受不稳定的训练.
研究的目的:
- 为了解决深度InfoMax (DIM) 方法的培训不稳定性.
- 为深度代表性学习提出一种更稳定的无监督学习方法.
- 增强DIM在AI中的实际应用性.
主要方法:
- 提出了瓦瑟斯坦基于距离的深度信息传输 (WDIM) 方法.
- 取代了DIM中的对抗网络,以Wasserstein距离进行稳定的培训.
- 在CIFAR10,CIFAR100和STL10数据集上评估WDIM,用于无监督分类.
主要成果:
- 与原来的DIM方法相比,WDIM显示了更好的训练稳定性.
- 在无监督学习过程中,WDIM实现了更快的模型融合.
- 拟议的WDIM方法保持了竞争力的分类准确性,与DIM相比.
结论:
- WDIM为无监督深度表示学习提供了稳定高效的替代方案.
- 该方法提高了训练深度学习模型的可靠性.
- WDIM为无监督图像分类的未来研究提供了一个有希望的方向.
相关概念视频
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
515
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
515
Mean Absolute Deviation
2.6K
The mean absolute deviation is also a measure of the variability of data in a sample. It is the absolute value of the average difference between the data values and the mean.
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
2.6K
Routh-Hurwitz Criterion II
256
In the application of the Routh-Hurwitz criterion, two specific scenarios can arise that complicate stability analysis.
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
256
Improving Translational Accuracy
10.8K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
10.8K
Wilcoxon Signed-Ranks Test for Matched Pairs
134
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
134
Weighted Mean
5.2K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
5.2K


