Post Hoc comparisons in fixed-effect ANOVA: A sequential fusion approach with calibrated error control

Post hoc comparisons following a significant analysis of variance (ANOVA) are typically conducted using pairwise methods such as Tukey's honestly significant difference or Holm-Bonferroni corrections. Although these procedures control family-wise error rates, they can produce nontransitive patterns of equivalence that do not correspond to any coherent partition of the factor levels. We propose a sequential fusion procedure that takes a modeling perspective: Starting from the saturated ANOVA model, the algorithm iteratively fuses the pair of groups with the most similar means, testing each fusion via a likelihood-ratio test using the ANOVA error term. A calibrated sequence of critical values ensures family-wise error rate control at the nominal level (e.g., 5%). Through extensive Monte Carlo simulations across balanced designs with 3-50 groups and 5-50 observations per group, we derive a simple approximation for the calibration factor as a function of the number of groups and sample size. Comparisons with Tukey's honestly significant difference, Holm-Bonferroni, Benjamini-Hochberg, and Scheffé procedures demonstrate that the fusion cascade achieves comparable or superior power while guaranteeing interpretational coherence: The output is always a single, transitive partition of groups. We illustrate the method's practical utility by reanalyzing data from Moreno and Mayer (2000) on multimedia learning, demonstrating how factorial designs are naturally handled by treating them as one-way ANOVA on cell means, with interaction structure revealed directly as a partition. Simulation studies confirm that the fusion cascade consistently identifies exact partitions under well-separated conditions, whereas pairwise methods produce nontransitive patterns in the majority of cases. (PsycInfo Database Record (c) 2026 APA, all rights reserved).