Related Experiment Video
Updated: Apr 27, 2026

An Integrated Workflow of Identification and Quantification on FDR Control-Based Untargeted Metabolome
Published on: September 20, 2022
POWER-ENHANCED MULTIPLE DECISION FUNCTIONS CONTROLLING FAMILY-WISE ERROR AND FALSE DISCOVERY RATES
Edsel A Peña1, Joshua D Habiger1, Wensong Wu1
1University of South Carolina, Columbia, Oklahoma State University and University of South Carolina, Columbia.
Abstract:
Improved procedures, in terms of smaller missed discovery rates (MDR), for performing multiple hypotheses testing with weak and strong control of the family-wise error rate (FWER) or the false discovery rate (FDR) are developed and studied. The improvement over existing procedures such as the Šidák procedure for FWER control and the Benjamini-Hochberg (BH) procedure for FDR control is achieved by exploiting possible differences in the powers of the individual tests. Results signal the need to take into account the powers of the individual tests and to have multiple hypotheses decision functions which are not limited to simply using the individual p-values, as is the case, for example, with the Šidák, Bonferroni, or BH procedures. They also enhance understanding of the role of the powers of individual tests, or more precisely the receiver operating characteristic (ROC) functions of decision processes, in the search for better multiple hypotheses testing procedures. A decision-theoretic framework is utilized, and through auxiliary randomizers the procedures could be used with discrete or mixed-type data or with rank-based nonparametric tests. This is in contrast to existing p-value based procedures whose theoretical validity is contingent on each of these p-value statistics being stochastically equal to or greater than a standard uniform variable under the null hypothesis. Proposed procedures are relevant in the analysis of high-dimensional "large M, small n" data sets arising in the natural, physical, medical, economic and social sciences, whose generation and creation is accelerated by advances in high-throughput technology, notably, but not limited to, microarray technology.
Insights
New multiple hypotheses testing procedures reduce missed discovery rates (MDR) by considering individual test powers, improving upon existing methods for family-wise error rate (FWER) and false discovery rate (FDR) control.
Area of Science:
- Statistical methodology
- Biostatistics
- Data science
Background:
- Multiple hypotheses testing is crucial in analyzing complex datasets.
- Existing methods like Šidák and Benjamini-Hochberg (BH) control error rates but may not be optimal.
- High-throughput technologies generate large datasets requiring efficient statistical analysis.
Purpose of the Study:
- To develop improved procedures for multiple hypotheses testing.
- To achieve better control of family-wise error rate (FWER) and false discovery rate (FDR).
- To reduce the missed discovery rate (MDR).
Main Methods:
- Developed novel procedures exploiting differences in individual test powers.
- Utilized a decision-theoretic framework.
- Incorporated auxiliary randomizers for discrete or mixed-type data and nonparametric tests.
Main Results:
- Achieved smaller missed discovery rates (MDR) compared to existing procedures.
- Demonstrated the advantage of incorporating individual test powers into decision functions.
- Showcased the utility of receiver operating characteristic (ROC) functions in enhancing testing procedures.
Conclusions:
- Proposed procedures offer superior performance in multiple hypotheses testing.
- Considering individual test powers is essential for more effective error rate control.
- The methods are applicable to high-dimensional data analysis across various scientific fields.
Related Concept Videos
Types of Errors: Detection and Minimization
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Detection of Gross Error: The Q Test
Errors In Hypothesis Tests

