Local independence and residual covariance: a study of olympic figure skating ratings

Rasch fit analysis has focused on tests of global fit and tests of the fit of individual parameter estimates. Critics have noted that slight, but pervasive, patterns of misfit to a Rasch model within the data may escape detection using these approaches. These patterns contradict the Rasch axiom of local independence, and so degrade measurement and may bias measures. Misfit to a Rasch model is captured in the observation residuals. Traces of pervasive, but faint, secondary dimensions within the observations may be identified using factor analytic techniques. To illustrate these techniques, the ratings awarded during the Pairs Figure Skating competition at the 2002 Winter Olympic Games are examined. The intention is to detect analytically the patterns of rater bias admitted publicly after the event. It is seen that the one-parameter-at-a-time fit statistics and differential item functioning approaches fail to detect the crucial misfit patterns. Factor analytic methods do. In fact, the competition was held in two stages. Factor analytic techniques already detect the rater bias after the first stage. This suggests that remedial rater retraining or other rater-related actions could be taken before the final ratings are collected.