对于深度网络进行基准测试的实用概括度量
Mengqing Huang1, Hongchuan Yu2, Jianjun Zhang1
1National Centre for Computer Animation, Bournemouth University, Poole, BH12 5BB, UK.
Scientific reports
|March 22, 2025
概括
本研究介绍了一种实用度量来评估深度学习模型的概括性,发现它取决于准确性和数据多样性. 大多数现有的理论估计与实际测量有很差的相关性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 在深度学习模型中估计概括错误对于实际应用和理论验证都至关重要.
- 目前的研究缺乏标准化的方法来对深度网络泛化进行基准测试,并验证理论预测.
- 实际评估对于弥合理论估计与现实绩效之间的差距至关重要.
研究的目的:
- 引入一种实用的概括度量,用于对各种深度学习网络进行基准测试.
- 提出一种用于验证理论概括估计的新型试验台.
- 量化模型准确性,数据多样性和概括能力之间的关系.
主要方法:
- 开发一种新的实践概括度量.
- 为深度学习模型创建一个比较测试平台.
- 对拟议的指标与现有的理论概括估计进行比较分析.
主要成果:
- 在分类中的深度网络泛化受到分类准确性和未见数据多样性的影响.
- 拟议的指标量化了模型准确性和数据多样性,提供了直观的权衡评估.
- 大多数现有的理论概括估计表明与新测试台的实际测量有很差的相关性.
结论:
- 拟议的指标提供了对深度学习模型概括的定量评估.
- 当前的理论概括估计与实际表现之间存在重大差异.
- 这项工作突出了现有理论的局限性,并激励进一步研究更准确的概括评估方法.
相关概念视频
Generalization, Discrimination, and Extinction
375
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
375
Improving Translational Accuracy
8.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
8.5K
Mean Absolute Deviation
2.5K
The mean absolute deviation is also a measure of the variability of data in a sample. It is the absolute value of the average difference between the data values and the mean.
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
2.5K
Multiple Comparison Tests
3.8K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.8K
Testing a Claim about Standard Deviation
2.4K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.4K
Sensitivity, Specificity, and Predicted Value
157
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
157


