Regularized Bayesian calibration and scoring of the WD-FAB IRT model improves predictive performance over marginal...

Area of Science:

• Psychometrics
• Statistical modeling
• Educational measurement

Background:

• Item response theory (IRT) models, particularly the graded response model (GRM), are essential for quantifying individual traits from test responses.
• Statistical decisions in IRT model calibration and test scoring impact accuracy and interpretability.
• Common applications, like the Work Disability Functional Assessment Battery (WD-FAB), often require computationally tractable approximations for scoring.

Purpose of the Study:

• To evaluate the calibration and scoring performance of GRM implementations under common use-case conditions.
• To compare regularized Bayesian calibration against the empirical Bayesian marginal maximum likelihood procedure.
• To assess the predictive power of GRM ability estimates in forecasting response patterns.

Main Methods:

• Bayesian cross-validation was employed to assess model calibration and scoring.
• The study utilized response data from the WD-FAB collected for the National Institutes of Health.
• Evaluated GRM implementations focused on predictive accuracy of ability estimates on validation sets.

Main Results:

• Regularized Bayesian calibration of the GRM demonstrated superior performance over the marginal maximum likelihood procedure.
• The study identified that specific calibration methodologies significantly influence the predictive power of ability estimates.
• Compactly supported priors were motivated for improved test scoring.

Conclusions:

• Regularized Bayesian calibration offers enhanced predictive accuracy for GRM in psychometric assessments.
• The findings advocate for the use of regularization techniques in IRT model development and application.
• The research provides insights into optimizing test scoring for practical applications like the WD-FAB.