Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity

Area of Science:

• Computational Statistics
• Bayesian Inference

Background:

• Simulation-based calibration checking (SBC) is crucial for validating posterior distributions from computational models.
• Existing SBC methods have limitations in detecting certain classes of posterior distribution errors, such as when the posterior is indistinguishable from the prior.

Purpose of the Study:

• To introduce a novel variant of simulation-based calibration checking (SBC) that enhances the detection of errors in posterior distributions.
• To address limitations of previous SBC implementations, enabling the identification of a broader range of potential issues.

Main Methods:

• Development of a new SBC variant incorporating additional data-dependent test quantities.
• Theoretical analysis of the enhanced SBC method to understand its statistical underpinnings.
• Investigation of the joint likelihood of the data as a key test quantity.
• Numerical case studies using a multivariate normal distribution and an ordered simplex data type with Hamiltonian Monte Carlo.

Main Results:

• The proposed SBC variant can detect a wider range of posterior distribution problems compared to existing methods.
• The joint likelihood of the data is demonstrated as a powerful test quantity for SBC.
• Theoretical analysis provides a deeper understanding of SBC mechanisms.
• Case studies validate the practical utility and effectiveness of the new SBC approach.

Conclusions:

• The enhanced SBC variant offers a more comprehensive validation of posterior distributions in computational statistics.
• The inclusion of specific data-dependent test quantities significantly improves the diagnostic power of SBC.
• This work clarifies common misconceptions and provides a robust tool for Bayesian inference validation.