A survey of theory and methods of invariant item ordering

In many testing situations, ordering the items by difficulty is helpful in analysing the testing data; examples include intelligence testing, analysis of differential item functioning, person-fit analysis, and exploring hypotheses about the order in which cognitive operations are acquired by children. In each situation, interpretation and analysis are made easier if the items are ordered by difficulty in the same way for every individual taking the test, i.e. the item response functions do not cross. This is an invariant item ordering. In this paper we review a class of non-parametric unidimensional item response models in which the ordinal properties of items (and persons) can be studied, and survey both old and new methods for the investigation of invariant item ordering in empirical data sets. Our model formulation derives in particular from the work of Holland & Rosenbaum (1986), Junker (1993) and Mokken (1971). We survey methods based on the work of Mokken (1971), Rosenbaum (1987a, b), and Sijtsma & Meijer (1992), and we also discuss some new proposals for checking invariant item ordering. When violations are detected, these methods allow a rough assessment of where on the latent scale the item response functions cross. We also study similarities and differences between these various methods and provide guidelines for their use. Finally, the methods are illustrated with data from a developmental psychology experiment in which the ability to draw inferences about transitive relations is explored.