Metric Stability in Item Response Models.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Item parameter uncertainty in item response theory (IRT) can propagate to subsequent analyses, like individual trait score estimation.
• Existing methods often assume a fixed latent trait continuum, overlooking uncertainty in its location due to parameter estimation errors.
• This uncertainty implies variability in the metric itself, impacting the reliability of IRT models.

Purpose of the Study:

• To introduce a quantitative measure of metric stability in item response theory.
• To address the underappreciated uncertainty in the latent trait continuum's location stemming from item parameter estimation errors.
• To provide a novel tool for robust item response model evaluation.

Main Methods:

• Utilizing Ramsay's (1996) geometry of the latent trait metric to define and quantify metric stability.
• Developing a measure representing the sampling variability of the latent trait continuum.
• Illustrating the application and interpretation of metric stability through various examples.

Main Results:

• Metric stability quantifies the sampling variability of the latent trait continuum arising from item parameter estimation errors.
• The study demonstrates the relationship between metric stability and other IRT model evaluation metrics, such as test information and model fit.
• Metric stability aids in identifying stable regions of the latent trait continuum, informing sample size recommendations and model selection.

Conclusions:

• The proposed measure of metric stability offers a meaningful and interpretable way to evaluate item response models.
• Metric stability addresses the critical issue of latent trait metric uncertainty, enhancing the rigor of IRT analyses.
• This new metric provides valuable insights for researchers and practitioners using item response theory in diverse fields.