Estimating and Using Block Information in the Thurstonian IRT Model

Area of Science:

• Psychometrics
• Statistical Modeling
• Educational Measurement

Background:

• Multidimensional forced-choice (MFC) tests are increasingly popular but complex to construct.
• The Thurstonian item response model (Thurstonian IRT) is commonly used for scoring MFC tests with dominance items.
• Current frequentist approaches for Thurstonian IRT scoring neglect stochastic dependencies in pairwise comparisons.

Purpose of the Study:

• To develop a method for estimating Fisher information at the block level within the Thurstonian IRT model.
• To investigate the accuracy of standard errors using block information compared to traditional methods.
• To demonstrate how multidimensional block information can be summarized to aid MFC test construction.

Main Methods:

• Estimation of Fisher information on the block level for Thurstonian IRT models.
• Simulation studies to compare observed and expected standard errors based on block information.
• Analysis of the impact of neglecting local dependencies on standard error estimates.
• Summarization of multidimensional block information for practical test construction.

Main Results:

• Observed and expected standard errors based on block information showed similar accuracy.
• Neglecting local dependencies led to underestimated standard errors, except when using the maximum a posteriori estimator.
• Block information summaries exhibited small differences based on the outcome considered in simulation and empirical applications.

Conclusions:

• Estimating Fisher information at the block level provides an accurate method for scoring MFC tests.
• Block information accounting for dependencies aids in constructing more reliable MFC tests.
• The proposed method offers a valuable tool for psychometricians and test developers.