Rethinking the Interpretation of Item Discrimination and Factor Loadings

Factor loadings and item discrimination parameters play a key role in scale construction. A multitude of heuristics regarding their interpretation are hardwired into practice-for example, neglecting low loadings and assigning items to exactly one scale. We challenge the common sense interpretation of these parameters by providing counterexamples and general results which altogether cast doubt on our understanding of these parameters. In particular, we highlight the counterintuitive way in which the best prediction of a test taker's latent ability depends on the factor loadings. As a consequence, we emphasize that practitioners need to shift their focus from interpreting item discrimination parameters by their relative loading to an interpretation which incorporates the structure of the model-based latent ability estimate.