Comparing and weighting imperfect models using D-probabilities.

Area of Science:

• Statistics
• Machine Learning
• Bayesian Inference

Background:

• Traditional model weighting methods can be sensitive to prior choices and may not adequately handle model complexity.
• Assessing goodness-of-fit and comparing imperfect models are crucial in statistical analysis and machine learning.

Purpose of the Study:

• To propose a novel divergence-based method for assigning weights to statistical models, termed D-probabilities.
• To demonstrate the utility of D-probabilities in model comparison, goodness-of-fit assessment, and model aggregation.

Main Methods:

• Utilizing Kullback-Leibler divergence to evaluate parametric models against a nonparametric Bayesian reference.
• Developing analytic forms for D-probabilities in the context of linear model selection against a Gaussian process reference.

Main Results:

• D-probabilities provide a robust alternative to Bayesian model probabilities, showing less sensitivity to prior choices.
• The method inherently penalizes model complexity and assigns higher weights to a more diverse set of models.
• Analytic forms and probabilistic interpretations for D-probabilities are derived and illustrated.

Conclusions:

• D-probabilities offer a flexible and effective framework for model weighting, comparison, and aggregation.
• The approach is practical for implementation, as shown through simulations and a real-world ozone data application.