Score-based measurement invariance checks for Bayesian maximum-a-posteriori estimates in item response theory

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Score-based tests assess parameter invariance in Item Response Theory (IRT) models.
• Existing tests are primarily developed within a maximum likelihood (ML) framework.
• There is a need for analogous tests applicable to Bayesian (MAP) estimates and multiple-group IRT models.

Purpose of the Study:

• To propose and evaluate score-based statistical tests for Bayesian MAP estimates and multiple-group IRT models.
• To investigate the performance of these tests against differential item functioning (DIF) using categorical and continuous covariates.
• To compare two novel test families: one based on pooled variance and another on simulation.

Main Methods:

• Development of two families of statistical tests: pooled variance approximation and simulation-based approach.
• Evaluation through a simulation study examining sensitivity to DIF.
• Application to two- and three-parameter logistic IRT models.

Main Results:

• The pooled variance method demonstrated practical utility for both ML and MAP estimates.
• The simulation-based approach showed satisfactory results only with large sample sizes.
• Both methods were evaluated for their sensitivity to DIF with various person covariates.

Conclusions:

• The pooled variance method offers a practical approach for assessing parameter invariance in IRT, including Bayesian contexts.
• The simulation-based approach requires substantial sample sizes for reliable DIF detection.
• These findings contribute to robust psychometric analysis in IRT, particularly for complex models and estimation methods.