Empirical Bayes factors for common hypothesis tests.

Area of Science:

• Statistics
• Statistical Inference
• Bayesian Statistics

Background:

• Bayes factors struggle with vague prior knowledge for composite hypotheses due to limitations of improper and objective priors.
• Existing methods may yield subjectively unreasonable results or be difficult to interpret.
• A need exists for robust methods to quantify evidence in statistical hypothesis testing.

Purpose of the Study:

• To propose and evaluate the posterior Bayes factor as a solution for encoding vague prior knowledge in composite hypothesis testing.
• To develop an adjusted empirical Bayes factor that is comparable to proper Bayes factors.
• To establish an objective framework for interpreting statistical evidence and reconcile Bayesian and frequentist methodologies.

Main Methods:

• Revisiting the posterior Bayes factor, using the posterior distribution from the current data for Bayes factor calculation.
• Adjusting the posterior Bayes factor to mitigate bias when calibrated against proper Bayes factors.
• Developing test-based empirical Bayes factors for standard statistical tests and extending to multiple testing scenarios.
• Proposing an approximate empirical Bayes factor based on P-values and a logarithmic interpretation scale.

Main Results:

• The adjusted posterior Bayes factor provides an interpretable scale comparable to proper Bayes factors.
• For regular normal models, the bias in log scale is half the number of parameters.
• The empirical Bayes factor demonstrates a close relationship with the widely applicable information criterion (WAIC).
• An approximate empirical Bayes factor of 10p is proposed for situations with only P-values available.

Conclusions:

• The empirical Bayes factor offers a viable approach to handle vague prior knowledge in composite hypothesis testing.
• The proposed interpretation scale (log base 3.73) provides an objective measure of statistical evidence strength.
• This framework facilitates a compromise between Bayesian and frequentist statistical inference, enhancing practical applicability.