Related Experiment Video
Updated: May 7, 2026
06:35
Basics of Multivariate Analysis in Neuroimaging Data
Published on: July 24, 2010
Controlling the maximum familywise Type I error rate in analyses of multivariate experiments.
Kevin D Bird1, Dusan Hadzi-Pavlovic2
1School of Psychology, University of New South Wales.
Psychological Methods
|October 2, 2013
Summary
Standard multivariate analysis of variance (MANOVA) protected multiple-comparison procedures (MCPs) offer illusory Type I error control. Modified or single-stage MCPs nonconservatively control the maximum familywise error rate (MFWER) and should replace standard methods.
Area of Science:
- Statistics
- Experimental Design
- Psychometrics
Background:
- Multivariate experiments commonly use multistage multiple-comparison procedures (MCPs).
- These procedures prohibit univariate testing if the overall multivariate analysis of variance (MANOVA) test is not significant.
- The MANOVA test's primary role is Type I error rate control, but popular MANOVA-protected MCPs fail to control the maximum familywise error rate (MFWER).
Purpose of the Study:
- To demonstrate that standard MANOVA-protected MCPs provide illusory Type I error protection.
- To introduce modified protected MCPs and single-stage MCPs that control the MFWER.
- To advocate for the adoption of these improved MCPs.
Main Methods:
- Analysis of Type I error rates in standard MANOVA-protected MCPs.
- Development and evaluation of modified protected MCPs.
- Development and evaluation of single-stage MCPs for simultaneous confidence intervals on effect sizes.
Main Results:
- The MFWER associated with standard MANOVA-protected MCPs can be excessively large, rendering the MANOVA protection ineffective.
- Modified protected MCPs and single-stage MCPs can control the MFWER nonconservatively.
- These improved MCPs facilitate the construction of simultaneous confidence intervals for effect sizes.
Conclusions:
- Standard MANOVA-protected MCPs offer inadequate Type I error control.
- Modified protected and single-stage MCPs provide reliable MFWER control and enable effect size estimation.
- The ease of implementation justifies replacing standard procedures with these advanced MCPs.
Related Concept Videos
01:13Accuracy and Errors in Hypothesis Testing
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
01:10Bonferroni Test
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
01:14Errors In Hypothesis Tests
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
01:15Systematic Error: Methodological and Sampling Errors
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
01:16Statistical Hypothesis Testing
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...