An Examination of Sensitivity to Measurement Error in Rasch Residual-based Fit Statistics

The purpose of this paper is to examine the sensitivity of commonly used Rasch fit measures to different distributions of error in item responses. Using Monte Carlo methods, we generated 10 different measurement error conditions within the Rasch rating scale model or partial credit model, and we recorded the estimates of INFIT MNSQ, OUTFIT MNSQ, and person separation reliability for each error distribution condition. INFIT MNSQ and OUTFIT MNSQ were not sensitive to error distributions when the distribution was the same across items. When the error distribution varies across items, INFIT MNSQ and OUTFIT MNSQ detected items with higher levels of measurement error as potentially misfitting. The Rasch person separation reliability statistic was sensitive to varying levels of measurement error, as expected. Our findings have implications for the use of fit measures in diagnosing model misfit.