一种概率方法来预测大数据集上的分类器准确性,给出小试点数据
Ethan Harvey1, Wansu Chen2, David M Kent3
1Department of Computer Science, Tufts University, Medford, MA, USA.
Proceedings of machine learning research
|February 17, 2025
概括
本研究引入了高斯过程模型,用于预测随着数据大小的增加而提高分类器准确度的改进. 该模型提供了概率推断和不确定性评估,这对于数据驱动项目至关重要.
科学领域:
- 机器学习 机器学习
- 计算统计学 计算统计学
背景情况:
- 分类器的开发通常从有限的数据开始,并计划未来扩展.
- 从增加数据集大小来准确预测性能增长对于资源分配和项目规划至关重要.
研究的目的:
- 随着数据集大小的增长,提出一种用于对分类器性能指标进行概率推断的新方法.
- 通过纳入不确定性评估来解决现有的决定性推断方法的局限性.
主要方法:
- 开发高斯过程模型,以预测性能指标 (例如准确性) 作为数据集大小的函数.
- 在六个不同的数据集中使用错误,概率和覆盖率指标评估模型的性能.
主要成果:
- 建议的高斯过程模型提供了对分类器性能的可靠的概率推断.
- 该模型有效量化了与不同数据集大小的准确性预测相关的不确定性.
- 在六个数据集的经验评估表明了模型的稳定性和通用性.
结论:
- 与传统方法相比,高斯过程建模提供了一种优越的方法来推断分类器性能.
- 将不确定性纳入性能预测对于管理数据密集型机器学习项目的从业者来说至关重要.
- 该方法的开源性质使其在各种分类任务和数据模式中具有广泛的适用性.
相关概念视频
01:24Bootstrapping
577
The term "bootstrap" originated in the 19th century as a metaphor for self-improvement or achieving something independently, without external assistance. This concept extends to statistical bootstrapping, a self-contained method for estimating population parameters through resampling, even though it can be computationally intensive. Developed by the American statistician Dr. Bradley Efron in 1979, bootstrapping provides a robust way to perform inference when the original sample size is...
577
01:03Prediction Intervals
2.2K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.2K
01:19Survival Tree
52
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
52
01:24Testing a Claim about Population Proportion
3.3K
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
3.3K
01:13Accuracy and Errors in Hypothesis Testing
169
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
169
01:16Estimating Population Mean with Known Standard Deviation
8.2K
To construct a confidence interval for a single unknown population mean μ, where the population standard deviation is known, we need sample mean as an estimate for μ and we need the margin of error. Here, the margin of error (EBM) is called the error bound for a population mean (abbreviated EBM). The sample mean is the point estimate of the unknown population mean μ.
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
8.2K