Sources of Error in IRT Trait Estimation.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Item response theory (IRT) models describe the relationship between item characteristics, latent traits, and response probabilities.
• Trait score misestimation in IRT can arise from random error, estimation methods, item parameter errors, and model misspecification.

Purpose of the Study:

• To investigate the relative impact of different error sources on trait score bias and confidence interval coverage.
• To identify specific conditions under which model misspecification leads to inaccurate trait estimates.

Main Methods:

• Simulation study examining bias and coverage rates of trait scores under various error conditions.
• Analysis of item response functions (IRFs) to detect systematic conditional bias.
• Evaluation of model and item fit indices in identifying misspecification.

Main Results:

• Bias in trait scores was generally low, and coverage rates were acceptable for central trait scores, especially without strong Bayesian priors.
• Certain model misspecifications led to significant bias and poor coverage, particularly at the extremes of the latent trait.
• Conditionally biased IRFs, potentially undetected by fit indices, were identified as a source of biased trait estimates.

Conclusions:

• Model misspecification is a critical factor influencing the accuracy of trait score estimation in IRT.
• Biased trait estimates can result from systematic conditional bias in IRFs, which may not be apparent through standard fit diagnostics.
• The relationship between estimated and true latent traits can become nonlinear due to model misspecification, impacting score interpretation.