Infit and Outfit: Interpreting Statistical Significance and Magnitude of Misfit in Conjunction

In many areas of statistics it is common practice to present both a statistical significance test and an effect size. In contrast, for the Infit and Outfit indices of item misfit, it has historically been common to focus on either the mean square (MS; an index of the magnitude of misfit) or the statistical significance, but not both. If the statistical significance and effect size are to be used together, it is important not only that the Type I error rate matches the nominal alpha level, but also that, for any given magnitude of misfit, the expected value of the MS is independent of sample size. This study confirmed that the average MS for several simulated misfitting items was nearly the same for large and small samples, although necessarily the variance depended on sample size. Thus, if the item fit is statistically significant, the MS appears to be a reasonable index for judging the magnitude of the misfit in the sample, although one must recognize that the estimate of the magnitude will be less stable in small samples, as is true for all effect sizes.