为了更好地评估域外通用化
Duhun Hwang1, Suhyun Kang2, Moonjung Eo3
1Shopping Foundation Models Team, NAVER, South Korea.
概括
在域泛化 (DG) 研究中使用的平均值是不可靠的. 这项研究引入了最差+差距测量作为评估DG算法性能的更强大的替代方案.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 域泛化 (DG) 旨在创建在未见的数据分布上表现良好的算法.
- 平均指标通常用于比较DG算法,但其准确性值得怀疑.
- 现有的总局研究缺乏对平均措施的局限性进行彻底调查.
研究的目的:
- 在域泛化中研究平均量度的局限性.
- 提出和验证最差+差距的措施作为一个优越的替代方案.
- 提供一种更可靠的方法来评估DG算法性能.
主要方法:
- 理论分析以建立最差+差距衡量的基础,包括推导两个定理.
- 修改数据集的开发 (SR-CMNIST,C-Cats&Dogs,L-CIFAR10,PACS受损,VLCS受损) 以准确评估总局的绩效.
- 在拟议的最差+差距测量和传统的平均值测量之间进行了广泛的实验比较.
主要成果:
- 平均指标在接近真正的域泛化能力方面表现较差.
- 建议的最差+差距措施被证明是一个强大的,理论上支持的替代方案.
- 实验结果验证了最差+差距测量对平均测量的有效性.
结论:
- 平均指标不足以准确评估域泛化性能.
- 最糟糕+缺口措施为GD研究提供了更可靠和理论上更合理的方法.
- 这项研究为域名通用化的评估方法提供了关键的进步.
相关概念视频
Generalization, Discrimination, and Extinction
355
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
355
Quantifying and Rejecting Outliers: The Grubbs Test
1.4K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.4K
Detection of Gross Error: The Q Test
4.9K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
4.9K
Case Studies
11.5K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
11.5K
What Are Outliers?
3.6K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.6K
Testing a Claim about Standard Deviation
2.4K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.4K


