On Generalized Schürmann Entropy Estimators

Area of Science:

• Information Theory
• Statistical Estimation

Background:

• Estimating Shannon entropy for discrete distributions is challenging, especially with limited data.
• Existing methods often struggle with bias and variance issues in undersampled scenarios.

Purpose of the Study:

• To introduce a new class of Shannon entropy estimators.
• To address the limitations of current estimators in severely undersampled discrete distributions.

Main Methods:

• Generalizing previously proposed entropy estimators.
• Developing estimators with specific parameter choices to achieve desired statistical properties.

Main Results:

• The new estimators are unbiased and possess finite variance, even for severely undersampled distributions.
• Numerical tests demonstrate superior performance compared to existing estimators and exact values.
• A conflict with Bayesian estimators for mutual information was identified.

Conclusions:

• The proposed estimators offer a significant advancement in accurately quantifying Shannon entropy from sparse data.
• These findings challenge conventional understanding regarding bias and variance in entropy estimation.