Related Experiment Video
Updated: Jun 28, 2026
07:31
Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms
Published on: February 8, 2019
Partitioning to uncover conditions for permutation tests to control multiple testing error rates
Violeta Calian1, Dongmei Li, Jason C Hsu
1Science Institute, University of Iceland, Dunhaga 3, 107 Reykjavik, Iceland.
Biometrical Journal. Biometrische Zeitschrift
|October 22, 2008
Summary
Permutation multiple tests require specific assumptions to control the Familywise Error Rate (FWER). We found FWER control is maintained in linear models for Quantitative Trait Loci (QTL) analysis but may fail for gene expression or safety endpoints without further assumptions.
Area of Science:
- Statistics
- Genetics
- Biostatistics
Background:
- Permutation tests are widely used for multiple hypothesis testing.
- Controlling the Familywise Error Rate (FWER) is crucial for valid statistical inference.
- The validity of permutation tests can depend on assumptions about the joint distribution of data.
Purpose of the Study:
- To investigate the assumptions required for permutation multiple tests to control FWER.
- To examine the impact of marginal and joint distribution parameters on FWER control.
- To identify conditions under which permutation tests maintain FWER control in different statistical models.
Main Methods:
- Theoretical analysis of permutation test validity under various distributional assumptions.
- Examination of linear models with independent and identically distributed (i.i.d.) errors.
- Consideration of multivariate observations in the context of Quantitative Trait Loci (QTL) analysis, gene expression, and safety endpoints.
Main Results:
- Permutation testing validity is influenced by parameters in the joint distributions of observations.
- In linear models with i.i.d. errors (e.g., QTL analysis), FWER control is maintained if the test statistic has a specific form.
- Without assumptions linking marginal and joint distributions, permutation tests may not control FWER in gene expression or safety endpoint analyses.
Conclusions:
- Specific assumptions are essential for ensuring permutation multiple tests control FWER.
- The impact of distributional assumptions on FWER control varies depending on the statistical model and data type.
- Careful consideration of distributional properties is necessary for reliable results in complex multivariate analyses.
Related Concept Videos
01:10Bonferroni Test
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
01:24Testing a Claim about Population Proportion
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:13Multiple Comparison Tests
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
01:23Test for Homogeneity
The goodness–of–fit test can be used to decide whether a population fits a given distribution, but it will not suffice to decide whether two populations follow the same unknown distribution. A different test, called the test for homogeneity, can be used to conclude whether two populations have the same distribution. To calculate the test statistic for a test for homogeneity, follow the same procedure as with the test of independence. The hypotheses for the test for homogeneity can be stated as...
01:21Introduction to Test of Independence
In statistics, the term independence means that one can directly obtain the probability of any event involving both variables by multiplying their individual probabilities. Tests of independence are chi-square tests involving the use of a contingency table of observed (data) values.
The test statistic for a test of independence is similar to that of a goodness-of-fit test:
The test statistic for a test of independence is similar to that of a goodness-of-fit test:
01:13Accuracy and Errors in Hypothesis Testing
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...