A critical evaluation of the current "p-value controversy"

Area of Science:

• Biostatistics and Epidemiology
• Medical Research
• Statistical Sciences

Background:

• The American Statistical Association (ASA) initiated a discussion in 2016 regarding the over-reliance on p-values and its contribution to a reproducibility crisis in science.
• Concerns about p-value misuse and its impact on scientific reproducibility are prevalent across various research fields.
• Biostatistics and epidemiology, while sharing some concerns, are distinct due to established regulatory frameworks.

Purpose of the Study:

• To evaluate the severity of p-value-related issues in medical statistics compared to other scientific domains.
• To highlight the role of regulatory rules in mitigating p-value misuses within medical research.
• To advocate for a broader range of statistical inference methods beyond traditional p-value hypothesis testing.

Main Methods:

• Analysis of the ASA statement's relevance to medical statistics.
• Examination of regulatory guidelines governing statistical practices in medical research.
• Review of statistical methodologies beyond traditional null hypothesis significance testing.

Main Results:

• Many issues highlighted by the ASA are less severe in medical statistics due to robust regulatory oversight.
• Regulatory rules effectively ban common misuses of p-values in a significant portion of medical research.
• Current statistical practices risk ignoring advancements in inference methods beyond p-value-based testing.

Conclusions:

• Medical statistics benefits from regulatory frameworks that mitigate some p-value-related reproducibility issues.
• Reducing statistical analysis to only p-value tests under traditional null hypotheses overlooks significant methodological progress.
• A diverse repertoire of statistical inference methods can supplement and enhance current data analysis practices, addressing concerns raised by the ASA.