Bayes Factor vs. Posterior Predictive Model Assessment: Insights from Ordinal Constraints

Area of Science:

• Statistics and Probability
• Computational Statistics
• Model Selection and Assessment

Background:

• Effective statistical inference relies on appropriate model specification aligned with theoretical positions.
• Current inference methods may impose constraints that are not ideal for specific research questions.

Purpose of the Study:

• To evaluate the performance of posterior predictive model assessment methods (e.g., Watanabe-Aike Information Criterion [WAIC], Leave-One-Out Cross-Validation [LOO-CV]) in scenarios with nested models.
• To compare posterior predictive methods with Bayes factor model comparison when theoretical positions involve overlapping parameter spaces.

Main Methods:

• Investigated posterior predictive model assessment techniques under conditions where theoretical models correspond to nested parameter spaces.
• Examined the behavior of WAIC and LOO-CV when one model's parameter space is a subset of another's.
• Contrasted these methods with Bayes factor model comparison, which handles overlapping models effectively.

Main Results:

• Posterior predictive methods demonstrated failure in nested model scenarios, not favoring more constrained models even with compatible data.
• These methods necessitate partitioning parameter spaces into non-overlapping subspaces, potentially lacking theoretical justification.
• Bayes factor model comparison successfully accommodates overlapping models without imposing arbitrary partitions.

Conclusions:

• Posterior predictive approaches can be less desirable for substantive scientific applications due to forced model specifications.
• Researchers should consider the implications of model constraints imposed by posterior predictive methods.
• Bayes factor model comparison offers a more flexible alternative for scientific inference involving nested or overlapping theoretical models.