Reply to Rouder (2014): good frequentist properties raise confidence

Area of Science:

• Psychology
• Statistics
• Research Methodology

Background:

• Established psychological findings are questioned due to the ease of achieving statistical significance, even without real effects.
• The practice of stopping experiments based on results can artificially inflate significance.

Purpose of the Study:

• To investigate whether researchers should solely focus on Bayes factors, irrespective of experimental stopping rules.
• To explore the implications of stopping rules on the interpretation and reliability of Bayes factors in psychological research.

Main Methods:

• Review of existing literature on statistical significance, Bayes factors, and experimental stopping rules.
• Analysis of theoretical arguments and demonstrations concerning the influence of stopping rules on Bayes factors.

Main Results:

• While Bayes factors provide correct evidence, they can be influenced by experimental stopping rules.
• Good frequentist properties of statistical methods correlate with intuitive results and reduce future refutations.

Conclusions:

• Researchers should remain mindful of how experimental data is collected, not just the final Bayes factor.
• Considering frequentist properties alongside Bayes factors can bolster confidence in psychological research findings.