MERIT: Controlling Monte-Carlo error rate in large-scale Monte-Carlo hypothesis testing

Area of Science:

• Statistics
• Computational Biology
• Bioinformatics

Background:

• Large-scale hypothesis testing commonly uses Monte-Carlo (MC) p-values.
• MC error can lead to incorrect conclusions, especially for p-values near significance thresholds.
• Existing methods like Gandy-Hahn (GH) control MC error but are often too conservative.

Purpose of the Study:

• To introduce MERIT, a novel method for large-scale MC hypothesis testing.
• To demonstrate MERIT's statistical efficiency and control of MC error rate (MCER).
• To show MERIT outperforms the GH method in making correct decisions.

Main Methods:

• MERIT controls the MCER, similar to the GH method.
• Extensive simulation studies were conducted to evaluate MERIT's performance.
• The method was applied to analyze gene expression data from a prostate cancer study.

Main Results:

• MERIT effectively controls the MCER.
• MERIT demonstrates greater statistical efficiency than the GH method.
• MERIT yields more decisions that align with ideal p-values compared to GH.

Conclusions:

• MERIT provides a more statistically efficient approach to large-scale MC hypothesis testing.
• MERIT improves decision-making accuracy by controlling MCER.
• The method is applicable to real-world biological data analysis, such as gene expression studies.