Reporting trends of p values in the neurosurgical literature

The objective of this study was to evaluate the trends in reporting of p values in the neurosurgical literature from 1990 through 2017.

All abstracts from the Journal of Neurology, Neurosurgery, and Psychiatry (JNNP), Journal of Neurosurgery (JNS) collection (including Journal of Neurosurgery: Spine and Journal of Neurosurgery: Pediatrics), Neurosurgery (NS), and Journal of Neurotrauma (JNT) available on PubMed from 1990 through 2017 were retrieved. Automated text mining was performed to extract p values from relevant abstracts. Extracted p values were analyzed for temporal trends and characteristics.

The search yielded 47,889 relevant abstracts. A total of 34,324 p values were detected in 11,171 abstracts. Since 1990 there has been a steady, proportionate increase in the number of abstracts containing p values. There were average absolute year-on-year increases of 1.2% (95% CI 1.1%-1.3%; p < 0.001), 0.93% (95% CI 0.75%-1.1%; p < 0.001), 0.70% (95% CI 0.57%-0.83%; p < 0.001), and 0.35% (95% CI 0.095%-0.60%; p = 0.0091) of abstracts reporting p values in JNNP, JNS, NS, and JNT, respectively. There have also been average year-on-year increases of 0.045 (95% CI 0.031-0.059; p < 0.001), 0.052 (95% CI 0.037-0.066; p < 0.001), 0.042 (95% CI 0.030-0.054; p < 0.001), and 0.041 (95% CI 0.026-0.056; p < 0.001) p values reported per abstract for these respective journals. The distribution of p values showed a positive skew and strong clustering of values at rounded decimals (i.e., 0.01, 0.02, etc.). Between 83.2% and 89.8% of all reported p values were at or below the "significance" threshold of 0.05 (i.e., p ≤ 0.05).

Trends in reporting of p values and the distribution of p values suggest publication bias remains in the neurosurgical literature.