Examination of CIs in health and medical journals from 1976 to 2019: an observational study
Adrian Gerard Barnett1, Jonathan D Wren2,3
1Institute of Health and Biomedical Innovation, Queensland University of Technology, Kelvin Grove, Queensland, Australia a.barnett@qut.edu.au.
Objectives:
Previous research has shown clear biases in the distribution of published p values, with an excess below the 0.05 threshold due to a combination of p-hacking and publication bias. We aimed to examine the bias for statistical significance using published confidence intervals.
Design:
Observational study.
Setting:
Papers published in Medline since 1976.
Participants:
Over 968 000 confidence intervals extracted from abstracts and over 350 000 intervals extracted from the full-text.
Outcome Measures:
Cumulative distributions of lower and upper confidence interval limits for ratio estimates.
Results:
We found an excess of statistically significant results with a glut of lower intervals just above one and upper intervals just below 1. These excesses have not improved in recent years. The excesses did not appear in a set of over 100 000 confidence intervals that were not subject to p-hacking or publication bias.
Conclusions:
The huge excesses of published confidence intervals that are just below the statistically significant threshold are not statistically plausible. Large improvements in research practice are needed to provide more results that better reflect the truth.
More Related Videos
06:55Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
15:00Setup of Consumer Wearable Devices for Exposure and Health Monitoring in Population Studies
Published on: February 3, 2023
Related Concept Videos
Longitudinal Research
Cis-regulatory Sequences
Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
Longitudinal Studies
