Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Bonferroni Test01:10

Bonferroni Test

The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Errors In Hypothesis Tests01:14

Errors In Hypothesis Tests

When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
Multiple Comparison Tests01:13

Multiple Comparison Tests

Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Expected Frequencies in Goodness-of-Fit Tests01:19

Expected Frequencies in Goodness-of-Fit Tests

A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
Wald-Wolfowitz Runs Test II01:17

Wald-Wolfowitz Runs Test II

The Wald-Wolfowitz runs test, commonly referred to as the runs test, is a nonparametric test used to assess the randomness of ordered data. The test evaluates the number of runs, which are consecutive sequences of similar elements within the data. If the number of runs is significantly higher or lower than expected, the data is considered non-random, indicating a detectable pattern or structure.
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and 0s. In...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Analyzing Multivariate Repeated Measures Designs: A Comparison of Two Approximate Degrees of Freedom Procedures.

Multivariate behavioral research·2016
Same author

Repeated Measures Interaction Test with Aligned Ranks.

Multivariate behavioral research·2016
Same author

Testing Repeated Measures Hypotheses when Covariance Matrices are Heterogeneous: Revisiting the Robustness of the Welch-James Test.

Multivariate behavioral research·2016
Same author

Using Trimmed Means to Compare K Measures Corresponding to Two Independent Groups.

Multivariate behavioral research·2016
Same author

A Comparison of Data Analysis Strategies for Testing Omnibus Effects in Higher-Order Repeated Measures Designs.

Multivariate behavioral research·2016
Same author

MODERN REGRESSION METHODS THAT CAN SUBSTANTIALLY INCREASE POWER AND PROVIDE A MORE ACCURATE UNDERSTANDING OF ASSOCIATIONS.

European journal of personality·2013

Related Experiment Video

Updated: May 11, 2026

Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry
05:53

Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry

Published on: June 21, 2018

Error, power, and cluster separation rates of pairwise multiple testing procedures.

Juliet Popper Shaffer1, Rhonda K Kowalchuk, H J Keselman

  • 1Department of Statistics.

Psychological Methods
|May 8, 2013
PubMed
Summary

This study compares statistical methods for controlling familywise error rate and false discovery rate. Range-based methods offer more interpretable treatment separation patterns than p-value methods.

More Related Videos

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

Related Experiment Videos

Last Updated: May 11, 2026

Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry
05:53

Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry

Published on: June 21, 2018

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

Area of Science:

  • Statistics
  • Biostatistics
  • Experimental Design

Background:

  • Comparing multiple treatments involves controlling statistical error rates like familywise error rate (FWER) and false discovery rate (FDR).
  • Existing research often focuses on controlling either FWER or FDR, with limited direct comparison of methods across both.
  • Per-family error rate (PFER) is also considered, which can diverge from FWER with FDR control methods.

Purpose of the Study:

  • To compare statistical methods on their control of FWER, FDR, and PFER.
  • To introduce and evaluate new measures of interpretability based on treatment separation patterns.
  • To provide practical recommendations for method selection in multiple treatment comparisons.

Main Methods:

  • Evaluation of statistical methods for controlling FWER, FDR, and PFER.
  • Introduction of interpretability measures based on the pattern of treatment separation into non-overlapping sets.
  • Comparison of range-based (configural) methods versus individual p-value-based measures.

Main Results:

  • Range-based methods generally yield more interpretable patterns of treatment separation compared to individual p-value-based methods.
  • Differences in error rate control can lead to divergent outcomes, particularly when controlling FDR at low levels.
  • Power estimates based solely on the number of correct rejections may overlook crucial pattern information.

Conclusions:

  • Range-based methods are recommended for achieving interpretable treatment separation in multiple comparisons.
  • Statistical method selection should consider both error rate control and the interpretability of treatment groupings.
  • Practical recommendations are provided to aid researchers in choosing appropriate methods, even for complex analyses.