Confidence intervals permit, but do not guarantee, better inference than statistical significance testing.

Area of Science:

• Psychology
• Behavioral Neuroscience
• Medical Research

Background:

• Statistical significance testing (null hypothesis significance testing, NHST) can lead to misinterpretations of research findings.
• Confidence intervals (CIs) are an alternative statistical method for data interpretation.

Purpose of the Study:

• To compare how researchers interpret similar study results presented using NHST versus CIs.
• To assess the impact of referencing NHST when interpreting CIs.

Main Methods:

• An email survey was conducted with 330 authors from psychology, behavioral neuroscience, and medical journals.
• Participants were asked to interpret two fictitious studies with similar results, one statistically significant and one non-significant, presented via NHST or CIs.

Main Results:

• Interpretation of statistical results was generally poor across both NHST and CIs.
• Researchers interpreting CIs while referencing NHST were more likely (60%) to incorrectly conclude results conflicted.
• Researchers interpreting CIs without referencing NHST were highly likely (95%) to correctly conclude results were consistent.

Conclusions:

• Confidence intervals facilitate better statistical interpretation when not conflated with null hypothesis significance testing.
• Encouraging meta-analytic thinking and the use of CIs can improve statistical inference.
• Effective statistical reform requires researchers to interpret CIs independently of NHST.