Item Complexity: A Neglected Psychometric Feature of Test Items?

Area of Science:

• Psychometrics
• Educational Measurement
• Item Response Theory

Background:

• Item complexity is crucial in test development but often overlooked in psychometric modeling.
• Variability in item complexity can lead to asymmetry in item characteristic curves (ICCs).
• This asymmetry can significantly affect item response theory (IRT) applications, particularly those assuming interval-level latent proficiency and differing group means.

Purpose of the Study:

• To explore how item complexity variability manifests as ICC asymmetry.
• To investigate the impact of ICC asymmetry on IRT applications like vertical scaling and differential item functioning (DIF).
• To demonstrate how symmetric IRT models can lead to misinterpretations in the presence of ICC asymmetry.

Main Methods:

• Theoretical consideration of item complexity's effect on ICCs.
• Analysis of score gain deceleration in vertical scaling contexts.
• Examination of the correlation between DIF and item difficulty in verbal tests.
• Sensitivity analyses to assess the impact of ICC asymmetry on observed correlations.

Main Results:

• Complexity-induced positive ICC asymmetry can explain score gain deceleration in symmetric IRT models.
• Negative ICC asymmetry may explain the positive correlation between DIF and difficulty in certain tests.
• Symmetric IRT models can obscure systematic ICC asymmetry, leading to potential misinterpretations.

Conclusions:

• Item complexity-induced ICC asymmetry is a critical factor in psychometric modeling.
• Accounting for ICC asymmetry is essential for accurate IRT applications, especially in vertical scaling and DIF analysis.
• Further research should explore other applications where attending to ICC asymmetry may be beneficial.