Significance test for linear regression: how to test without P-values?

Area of Science:

• Statistics
• Econometrics
• Scientific Research Methodology

Background:

• The 2016 American Statistician Association discussion highlighted the misuse of p-values in scientific research.
• Researchers, particularly economists, may misinterpret guidelines on significance testing and p-value usage.
• There is a need to re-evaluate the reliability of p-values and explore alternative hypothesis testing methods.

Purpose of the Study:

• To re-examine the accuracy and reliability of p-values in statistical inference.
• To introduce and evaluate alternative methods for hypothesis testing, including Minimum Bayes Factors and Belief functions.
• To compare the performance of p-values with alternative approaches under different conditions.

Main Methods:

• A simulation study was conducted to investigate the reliability of p-values.
• Existing approaches, Minimum Bayes Factors (MBFs) and Belief functions, were introduced as potential replacements for p-values.
• The accuracy of a plausibility approach was compared against traditional p-values for decisions about the null hypothesis.

Main Results:

• Simulation results confirmed that p-values can be unreliable in certain scenarios.
• The proposed alternative approaches demonstrated utility as substitutes for p-values in statistical inference.
• The plausibility approach showed higher accuracy than p-values when the null hypothesis was true.
• Minimum Bayes Factors (MBFs) provided more reliable results than other methods when the null hypothesis was false.

Conclusions:

• P-values exhibit limitations in reliability, necessitating the exploration of alternative statistical tools.
• Minimum Bayes Factors and Belief functions show promise as more dependable methods for hypothesis testing.
• The study advocates for the adoption of more robust statistical approaches to enhance the validity of scientific conclusions.