Sequential Tests of Multiple Hypotheses Controlling False Discovery and Nondiscovery Rates

Area of Science:

• Statistics
• Data Science
• Machine Learning

Background:

• Sequential data analysis presents challenges in controlling statistical error rates.
• Existing methods often require strong assumptions about data distribution or independence.
• Simultaneous control of false discovery rate (FDR) and false nondisclosure rate (FNR) is crucial for reliable streaming data interpretation.

Purpose of the Study:

• To develop a general and flexible procedure for multiple hypothesis testing on sequential data.
• To simultaneously control both the false discovery rate (FDR) and false nondisclosure rate (FNR).
• To minimize assumptions regarding data stream characteristics like distribution, dimension, and dependence.

Main Methods:

• The proposed procedure extends the Benjamini and Hochberg (1995) fixed sample size method to sequential data.
• It requires only a test statistic for each data stream that controls type I and II error probabilities.
• No assumptions are needed about the joint distribution of statistics or data streams, accommodating dependent and heterogeneous data.

Main Results:

• The procedure guarantees simultaneous control of FDR and FNR for sequential hypothesis testing.
• It is applicable to various sampling schemes including sequential, group sequential, and truncated designs.
• The method is proven to maintain error control under minimal assumptions.

Conclusions:

• The developed procedure offers a robust and adaptable framework for hypothesis testing in streaming data environments.
• It provides simultaneous FDR and FNR control, enhancing the reliability of findings from sequential analyses.
• This method represents a significant advancement for statistical inference with complex, evolving datasets.