A Bayesian Random Block Item Response Theory Model for Forced-Choice Formats

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Traditional item response theory (IRT) models struggle with local dependence in forced-choice formats.
• Testlet models provide a framework for addressing item dependence within blocks.

Purpose of the Study:

• To propose the Bayesian random block IRT (BRB IRT) model for analyzing forced-choice item blocks.
• To evaluate the performance of the BRB IRT model in parameter estimation and trait screening.
• To explore the model's adaptability for more complex measurement scenarios.

Main Methods:

• Developed the BRB IRT model incorporating a random block effect to handle local dependence.
• Utilized Markov Chain Monte Carlo (MCMC) for simultaneous estimation of item and trait parameters.
• Conducted simulation studies and an empirical application comparing BRB IRT with the Thurstonian IRT model.

Main Results:

• The BRB IRT model demonstrated good performance in estimating item and trait parameters.
• The model effectively screened individuals with low scores on target traits.
• Model performance was contingent on item block composition, requiring negatively keyed items.
• BRB IRT showed equivalent performance to the Thurstonian IRT model in empirical application.

Conclusions:

• The BRB IRT model is a viable and effective tool for analyzing forced-choice data.
• The inclusion of negatively keyed items is crucial for optimal model performance.
• The BRB IRT model offers a flexible foundation for advanced psychometric modeling, such as incorporating covariates like gender.