Is There Evidence of P-Hacking in Imaging Research?

Area of Science:

• Medical Imaging
• Scientific Research Integrity

Background:

• P-hacking, the practice of conducting selective analyses until statistical significance is achieved, is a concern across scientific disciplines.
• Ensuring research integrity is crucial for reliable scientific advancements.

Purpose of the Study:

• To investigate the prevalence of p-hacking within the field of imaging research.
• To analyze trends in p-hacking over time in published imaging studies.

Main Methods:

• A large-scale text-mining analysis of over 967,000 abstracts from 136 imaging journals (1972-2021).
• Statistical evaluation using p-curves and binomial tests on randomly sampled P-values to detect clustering below the 0.05 significance threshold.
• Analysis divided into 10-year periods to track potential evolution of p-hacking practices.

Main Results:

• No significant concentration of P-values just below 0.05 was observed, indicating a lack of evidence for p-hacking.
• P-values were more frequently found in the lower range (0.04-0.045) than just below 0.05 (0.045-0.05).
• Time-trend analysis across five decades did not reveal any evidence of p-hacking.

Conclusions:

• The study did not find evidence of p-hacking in imaging research abstracts published since 1972.
• Acknowledges limitations in detecting all forms of p-hacking and suggests other biases like publication bias may still be present.