Related Experiment Videos
A note on controlling the number of false positives
Edward L Korn1, Boris Freidlin
1Biometric Research Branch, National Cancer Institute, Bethesda, MD 20852-7434, USA. korne@ctep.nci.nih.gov
Biometrics
|September 29, 2007
Summary
A Bonferroni-type procedure for controlling false positives has limited statistical power. Multivariate permutation tests offer a more powerful alternative for analyzing experimental data, particularly in genomics research.
Area of Science:
- Statistics
- Bioinformatics
- Genomics
Background:
- The Bonferroni-type procedure by Lehmann and Romano controls the probability of exceeding a specified number of false positives.
- This method is commonly used in statistical analysis to manage Type I errors.
Purpose of the Study:
- To compare the statistical power of the Bonferroni-type procedure with multivariate permutation tests.
- To highlight the advantages of permutation tests in specific experimental designs.
Main Methods:
- The study evaluates a Bonferroni-type procedure for bounding false positive probabilities.
- A multivariate permutation test is proposed as a more powerful alternative.
- Gene expression microarray data from breast cancer tumors is used as a case study.
Main Results:
- The Bonferroni-type procedure demonstrates poor statistical power when compared to multivariate permutation tests.
- Multivariate permutation tests show superior power in experimental designs amenable to permutation analysis.
Conclusions:
- For experimental designs suitable for permutation testing, multivariate permutation tests are recommended over Bonferroni-type procedures due to higher statistical power.
- The findings are particularly relevant for analyzing complex biological data, such as gene expression microarrays.
Related Concept Videos
01:13Accuracy and Errors in Hypothesis Testing
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
01:24Testing a Claim about Population Proportion
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:12Types of Errors: Detection and Minimization
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
01:10Bonferroni Test
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
01:13Sensitivity, Specificity, and Predicted Value
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
01:13Controls in Experiments
When conducting an experiment, it is crucial to have control to reduce bias and accurately measure the dependent variables. It also marks the results more reliable. Controls are elements in an experiment that have the same characteristics as the treatment groups but are not affected by the independent variable. By sorting these data into control and experimental conditions, the relationship between the dependent and independent variables can be drawn. A randomized experiment always includes a...