Using Bayesian Nonparametric Item Response Function Estimation to Check Parametric Model Fit.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Parametric item response theory (IRT) models often violate the logistic function assumption for item response functions (IRFs).
• Checking for violations of these assumptions is crucial for accurate measurement.
• Nonparametric IRT methods offer alternatives for assessing model fit.

Purpose of the Study:

• To evaluate the performance of a Bayesian nonparametric method for assessing IRF fit in parametric IRT models.
• To compare this Bayesian approach with the existing bootstrapping nonparametric method for mixed-format tests.
• To identify the strengths and weaknesses of each method under various simulation conditions.

Main Methods:

• Utilized Bayesian nonparametric estimation with posterior predictive model checking.
• Accounted for parameter estimation uncertainty within a Bayesian framework.
• Compared performance against a bootstrapping nonparametric method using simulation studies and real-data analysis.

Main Results:

• The Bayesian nonparametric method demonstrated higher power in detecting misfit items with large sample sizes.
• It exhibited lower Type I error rates compared to the bootstrapping method, particularly for nonmonotonic items.
• Both simulation and real-data studies successfully identified dichotomous and polytomous misfit items, indicating their location and magnitude.

Conclusions:

• The Bayesian nonparametric method is a powerful tool for assessing the fit of parametric IRT models, especially for mixed-format tests.
• This approach provides a robust alternative to traditional methods, offering improved detection of model misfit.
• The identified misfit items and their characteristics can inform test development and refinement.