关于从多维样本数据中准确估计信息理论量
Manuel Álvarez Chaves1, Hoshin V Gupta2, Uwe Ehret3
1Stuttgart Center for Simulation Science, Cluster of Excellence EXC 2075, University of Stuttgart, 70569 Stuttgart, Germany.
Entropy (Basel, Switzerland)
|May 24, 2024
概括
从连续数据中估计信息理论量是具有挑战性的. k-最近邻居 (k-NN) 方法通常优于内核密度估计和直方图,特别是有足够的数据,提供更好的准确性和效率.
科学领域:
- 数据科学数据科学数据科学
- 信息理论 信息理论
- 计算统计学 计算统计学
背景情况:
- 从连续数据中估计信息理论量 (例如,,相互信息) 是至关重要的,但由于需要高维度准确的概率密度函数估计,因此通常具有计算挑战性.
- 现有的方法,如核密度估计 (KDE),对于高维数据可能不可靠或不可行,因此需要探索替代方法.
研究的目的:
- 系统地比较内核密度估计 (KDE),内置频率 (历史图) 和k-最近邻居 (k-NN) 的性能,以从连续数据中估计信息理论量.
- 在各种数据分布,维度 (1-10),样本大小和超参数中评估这些方法,评估准确性,计算效率和实施挑战.
主要方法:
- 从不同形状和尺寸的分布中生成合成数据样本.
- 估计的,库尔巴克-莱布勒分歧,以及使用KDE,直方图和k-NN的相互信息.
- 将估计结果与封闭式解或数值积分作为参考进行比较.
- 基于估计准确性,计算时间和实现复杂性的评估性能.
主要成果:
- k-最近邻居 (k-NN) 估计方法在大多数评估指标中表现出卓越的性能,包括算法实现,计算效率和估计准确性,特别是当有足够的数据可用时.
- 核密度估计 (KDE) 和基于历史图的方法显示出局限性,特别是在更高的维度或稀疏的数据.
- 性能因数据特征,样本大小和估计的特定信息理论量而有所不同.
结论:
- 推k-NN方法作为一种强大而高效的技术,用于从连续数据中估计信息理论量,为高维应用提供实用解决方案.
- 估计方法的选择应考虑具体的数据特征,目标信息理论量和可用的样本大小.
- 开发了一个开源的Python 3工具箱,以促进这些方法的应用,促进信息理论量在跨学科数据分析中的更广泛使用.
相关概念视频
Estimation of the Physical Quantities
4.2K
On many occasions, physicists, other scientists, and engineers need to make estimates of a particular quantity. These are sometimes referred to as guesstimates, order-of-magnitude approximations, back-of-the-envelope calculations, or Fermi calculations. The physicist Enrico Fermi was famous for his ability to estimate various kinds of data with surprising precision. Estimating does not mean guessing a number or a formula at random. Instead, estimation means using prior experience and sound...
4.2K
What are Estimates?
5.0K
It isn't easy to measure a parameter such as the mean height or the mean weight of a population. So, we draw samples from the population and calculate the mean height or mean weight of the individuals in the sample. This sample data acts as a representative measure of the population parameter. These sample statistics are known as estimates.
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
5.0K
Distributions to Estimate Population Parameter
4.1K
The accurate values of population parameters such as population proportion, population mean, and population standard deviation (or variance) are usually unknown. These are fixed values that can only be estimated from the data collected from the samples. The estimates of each of these parameters are sample proportion, the sample mean, and sample standard deviation (or variance). To obtain the values of these sample statistics, data are required that have particular distribution and central...
4.1K
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
481
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
481
Estimating Population Mean with Unknown Standard Deviation
7.7K
In practice, we rarely know the population standard deviation. In the past, when the sample size was large, this did not present a problem to statisticians. They used the sample standard deviation s as an estimate for σ and proceeded as before to calculate a confidence interval with close enough results. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
William S. Gosset (1876–1937) of the...
William S. Gosset (1876–1937) of the...
7.7K
Sample Size Calculation
3.3K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.3K


