A comparison of multiple testing adjustment methods with block-correlation positively-dependent tests.

Area of Science:

• Statistics
• Bioinformatics
• Genomics

Background:

• High-dimensional data analysis involves numerous simultaneous hypothesis tests, increasing the risk of false positives.
• Controlling error rates (family-wise error rate or false discovery rate) is essential for reliable conclusions.
• Common adjustment methods often assume test independence, which may not hold in practice.

Purpose of the Study:

• To evaluate the performance of common multiplicity adjustment methods under dependent test statistics.
• To assess the impact of varying degrees of block-correlation positive dependence on these methods.
• To provide guidance on selecting appropriate methods for complex, high-dimensional datasets.

Main Methods:

• A simulation study was conducted to compare several widely used multiplicity adjustment techniques.
• The simulations incorporated varying levels of block-correlation positive dependence among hypothesis tests.
• Performance was evaluated based on error control and statistical power.

Main Results:

• The performance of different adjustment methods varied significantly under dependent test conditions.
• Methods assuming independence showed reduced effectiveness in controlling error rates when tests were correlated.
• Some methods demonstrated better robustness to positive dependence, maintaining acceptable error control and power.

Conclusions:

• The assumption of independence in multiple hypothesis testing adjustment methods can lead to inflated type I errors in high-dimensional data with dependent tests.
• Careful consideration of test dependence is crucial when selecting multiplicity adjustment strategies.
• Simulation studies are vital for understanding method performance in realistic, complex data scenarios.