Related Experiment Videos
P-values and confidence intervals: two sides of the same unsatisfactory coin
1Yale University School of Medicine, New Haven, Connecticut 06510, USA.
Journal of Clinical Epidemiology
|April 16, 1998
Summary
Statistical significance, determined by P-values or confidence intervals, relies on probability, not the magnitude of findings. Developing methods to assess substantive importance is a key scientific challenge.
Area of Science:
- Statistical methodology
- Scientific inference
Background:
- P-values and confidence intervals are standard tools for assessing statistical significance.
- An alpha level is chosen to control the probability of chance findings.
Purpose of the Study:
- To highlight the limitations of traditional statistical significance testing.
- To emphasize the need for evaluating substantive importance beyond probabilistic inference.
Main Methods:
- Review of the principles behind P-values and confidence intervals.
- Analysis of the interpretation of "statistical significance".
Main Results:
- Both P-value and confidence interval methods rely on probability and can be misleading if data assumptions are violated.
- Current methods infer significance from probability, neglecting the magnitude of observed effects.
Conclusions:
- A century of probability-based significance has overlooked substantive importance.
- Developing principles for assessing what constitutes a "big" or "small" effect is a critical scientific challenge.