Rediscovery of Good-Turing estimators via Bayesian nonparametrics

Area of Science:

• Statistical ecology
• Bioinformatics
• Machine learning
• Genetics

Background:

• Estimating discovery probabilities is crucial in various fields like genetics and machine learning.
• Both frequentist and Bayesian statistical methods have been proposed for this estimation.

Purpose of the Study:

• To investigate the relationship between the Good-Turing approach and a recent Bayesian nonparametric approach.
• To demonstrate the asymptotic equivalence of these estimators under specific prior assumptions.

Main Methods:

• Comparison of the Good-Turing frequentist nonparametric approach with a Bayesian nonparametric approach.
• Utilizing a two-parameter Poisson-Dirichlet prior for analysis.
• Developing methodology for deriving credible intervals for Bayesian estimators.

Main Results:

• Bayesian nonparametric estimators are asymptotically equivalent to smoothed Good-Turing estimators for large sample sizes.
• A novel methodology for deriving exact and asymptotic credible intervals was introduced.
• The methodology was validated through simulations and analysis of Expressed Sequence Tags data.

Conclusions:

• The study establishes a significant link between classical and modern statistical estimation techniques.
• The findings provide a theoretical foundation and practical tools for estimating discovery probabilities.
• The research has implications for fields requiring accurate estimation of novel discoveries from data.