Expected p-values in light of an ROC curve analysis applied to optimal multiple testing procedures.

Area of Science:

• Statistical methodology
• Biostatistics
• Decision theory

Background:

• P-values are commonly used in statistical inference but their stochastic nature is often overlooked.
• Neglecting the variability of p-values can lead to incorrect conclusions in data analysis.
• Evaluating the performance of statistical tests and identifying factor associations is challenging due to p-value stochasticity.

Purpose of the Study:

• To introduce the expected p-value (EPV) as a robust measure for assessing the performance of statistical decision-making rules.
• To integrate EPV analysis within the established receiver operating characteristic (ROC) curve framework.
• To develop novel methods for constructing and optimizing statistical tests.

Main Methods:

• Utilizing ROC curve analysis to visualize and evaluate the properties of statistical testing mechanisms, including partial EPVs.
• Developing optimal statistical tests by minimizing EPVs.
• Creating new methods for combining multiple test statistics effectively.
• Applying the EPV/ROC framework to a myocardial infarction dataset.

Main Results:

• Demonstrated that the EPV-based approach maximizes the integrated power of testing algorithms across various significance levels.
• Established a new methodology for investigating and constructing statistical decision-making procedures.
• Successfully constructed an optimal test using the EPV/ROC technique for disease data analysis.
• Showcased the utility of the EPV/ROC technique for evaluating and comparing decision-making procedures.

Conclusions:

• The EPV, when analyzed within an ROC framework, provides an efficient methodology for statistical decision-making.
• This approach offers significant advantages in evaluating, constructing, and optimizing statistical tests.
• The EPV/ROC technique is valuable for practical applications in biostatistics and other data-driven fields.