The role of the p-value in the multitesting problem.

Area of Science:

• Statistics
• Bioinformatics
• Computational Biology

Background:

• Modern scientific research generates vast quantitative data, necessitating the simultaneous testing of numerous hypotheses.
• Misleading statistical conclusions from hypothesis testing contribute to the reproducibility crisis in science.
• The standard interpretation of p-values is often lost in massive multitesting scenarios.

Purpose of the Study:

• To revisit controversies surrounding statistical hypothesis testing implications.
• To analyze the role of p-values in massive multitesting and their diminished probabilistic interpretation.
• To introduce Receiver Operating Characteristic (ROC) curves as a valuable tool for large-scale multitesting.

Main Methods:

• Revisiting statistical hypothesis testing controversies.
• Analyzing the p-value's role in massive multitesting.
• Utilizing an analogy between hypothesis testing and diagnostic processes.
• Introducing and applying Receiver Operating Characteristic (ROC) curves.
• Illustrating concepts with the Hedenfalk dataset.

Main Results:

• P-values lose their standard probabilistic interpretation in massive multitesting.
• The analogy with diagnostic processes highlights limitations of p-value interpretation.
• Receiver Operating Characteristic (ROC) curves offer a more robust approach for large-scale multitesting.

Conclusions:

• Rethinking the application and interpretation of p-values in the era of big data is crucial.
• Receiver Operating Characteristic (ROC) curves provide a valuable framework for navigating the challenges of massive multitesting.
• Adopting advanced statistical tools like ROC curves can help mitigate the reproducibility crisis.