Probabilistic Pairwise Model Comparisons Based on Bootstrap Estimators of the Kullback-Leibler Discrepancy.

Area of Science:

• Statistics
• Econometrics
• Machine Learning

Background:

• Classical hypothesis testing has limitations, requiring nested models and a true data-generating model.
• Discrepancy measures offer an alternative for model selection without these strict assumptions.

Purpose of the Study:

• To utilize a bootstrap approximation of the Kullback-Leibler discrepancy (BD) for model selection.
• To estimate the probability that a null model is closer to the true data-generating model than an alternative model.
• To propose and evaluate bias corrections for the BD estimator.

Main Methods:

• Bootstrap approximation of the Kullback-Leibler discrepancy.
• Bias correction methods: bootstrap-based correction and addition of the number of parameters.
• Exploration of estimator behavior in various model comparison scenarios.

Main Results:

• The bootstrap approximation (BD) provides a method to estimate model discrepancy probabilities.
• Proposed bias corrections enhance the accuracy of the BD estimator.
• The effectiveness of corrections is demonstrated across different model comparison settings.

Conclusions:

• The corrected BD estimator offers a robust approach to model selection, overcoming limitations of classical hypothesis testing.
• This method provides a valuable tool for researchers when comparing non-nested models or when the true data-generating model is unknown.
• Further exploration of bias correction techniques can refine model comparison methodologies.