A Bayes factor meta-analysis of Bem's ESP claim

Area of Science:

• Psychology
• Statistics
• Bayesian inference

Background:

• P-values are criticized for not accurately reflecting evidence in scientific data.
• Previous Bayes factor assessments of extrasensory perception (ESP) data analyzed experiments in isolation.
• A critique of the limitations of existing Bayes factor methods for multi-experiment analysis.

Purpose of the Study:

• To develop a meta-analytic Bayes factor for assessing evidence across multiple experiments.
• To re-evaluate the evidence for ESP presented by Bem (2011) using the new method.
• To provide a framework for updating beliefs based on cumulative experimental data.

Main Methods:

• Development of a novel meta-analytic Bayes factor.
• Application of the meta-analytic Bayes factor to Bem's (2011) ESP experimental data.
• Comparison of results with previous single-experiment Bayes factor analyses.

Main Results:

• The meta-analytic Bayes factor indicated slight evidence for ESP with neutral (Bayes factor = 3.23) and erotic (Bayes factor = 1.57) stimuli.
• Moderate evidence was found for ESP with emotionally valenced non-erotic stimuli (Bayes factor ≈ 40).
• The proposed method allows for a more accurate assessment of total evidence across multiple studies.

Conclusions:

• The meta-analytic Bayes factor provides a robust method for evaluating cumulative evidence in scientific research.
• While some evidence for ESP exists, it is not strong enough to overcome scientific skepticism.
• The findings highlight the importance of appropriate statistical methods for analyzing multi-experiment datasets.