Related Experiment Video
Updated: Jun 25, 2026
10:02
Assessment of Spontaneous Alternation, Novel Object Recognition and Limb Clasping in Transgenic Mouse Models of Amyloid-β and Tau Neuropathology
Published on: May 28, 2017
Comparisons of methods for multiple hypothesis testing in neuropsychological research.
Richard E Blakesley1, Sati Mazumdar, Mary Amanda Dew
1Department of Biostatistics, University of Pittsburgh, PA, USA. reb18@pitt.edu
Neuropsychology
|March 4, 2009
Summary
Controlling statistical errors in hypothesis testing with multiple outcomes is crucial. This study recommends specific methods like Hochberg, Hommel, and minP for maximizing power in correlated data analysis.
Area of Science:
- Statistics
- Neuropsychology
Background:
- Multiple outcome hypothesis testing risks Type I error inflation, reducing statistical power.
- Correlated outcomes complicate maintaining pre-set Type I error levels in research.
- Neuropsychological research frequently involves multiple, interrelated assessment measures.
Purpose of the Study:
- To develop a multiple hypothesis testing strategy that maximizes statistical power while controlling Type I error.
- To compare the performance of different p-value adjustment methods under varying correlation conditions.
Main Methods:
- Sensitivity analysis using a neuropsychological dataset.
- Simulation study to assess method robustness with diverse correlation patterns and magnitudes.
Main Results:
- Hochberg and Hommel methods are recommended for mildly correlated outcomes.
- Step-down minP method is advised for highly correlated outcomes.
- Caveats exist regarding software implementation of these methods.
Conclusions:
- The study provides evidence-based recommendations for selecting appropriate multiple hypothesis testing methods.
- Choosing the right method depends on the correlation structure of the data.
- Optimizing statistical power while controlling Type I error is achievable with appropriate adjustments.
Related Concept Videos
01:13Multiple Comparison Tests
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
01:10Bonferroni Test
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
01:14Decision Making: Traditional Method
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
01:16Statistical Hypothesis Testing
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
01:11Types of Hypothesis Testing
There are three types of hypothesis tests: right-tailed, left-tailed, and two-tailed.
When the null and alternative hypotheses are stated, it is observed that the null hypothesis is a neutral statement against which the alternative hypothesis is tested. The alternative hypothesis is a claim that instead has a certain direction. If the null hypothesis claims that p = 0.5, the alternative hypothesis would be an opposing statement to this and can be put either p > 0.5, p < 0.5, or p ≠ 0.5.
When the null and alternative hypotheses are stated, it is observed that the null hypothesis is a neutral statement against which the alternative hypothesis is tested. The alternative hypothesis is a claim that instead has a certain direction. If the null hypothesis claims that p = 0.5, the alternative hypothesis would be an opposing statement to this and can be put either p > 0.5, p < 0.5, or p ≠ 0.5.
01:13Accuracy and Errors in Hypothesis Testing
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...