Can Researchers' Personal Characteristics Shape Their Statistical Inferences?

Area of Science:

• Psychology
• Statistical Methods

Background:

• Subjective researcher judgments can impact statistical results.
• Bayesian hypothesis testing requires specifying an expected effect size (prior mean).
• The prior mean reflects confidence in studying a large effect.

Purpose of the Study:

• To investigate the relationship between researcher self-confidence and prior mean selection in Bayesian hypothesis testing.
• To explore potential gender differences in prior mean selection.

Main Methods:

• Two preregistered studies were conducted.
• Over 900 active psychology researchers participated.
• Participants' self-confidence and prior mean selections were analyzed.

Main Results:

• Higher researcher self-confidence was associated with selecting larger prior means.
• Suggestive evidence indicated men may choose larger prior means than women, linked to self-confidence differences.

Conclusions:

• Researchers' personal characteristics, like self-confidence, can shape statistical results in Bayesian hypothesis testing.
• This highlights a potential source of bias in scientific findings using this method.