Bayesian item fit analysis for unidimensional item response theory models.

Area of Science:

• Psychometrics
• Statistical Modeling

Background:

• Assessing item fit in unidimensional item response theory (IRT) models for dichotomous items is crucial but lacks universally accepted diagnostics.
• Existing methods for item fit evaluation present limitations, necessitating further research and development.

Purpose of the Study:

• To introduce and evaluate novel Bayesian posterior predictive model-checking methods for assessing item fit in unidimensional IRT models with dichotomous items.
• To propose an item fit plot and posterior predictive p-values for established item fit statistics.

Main Methods:

• Utilized the posterior predictive model-checking method, a Bayesian approach for model assessment.
• Developed an item fit plot comparing observed and predicted proportion-correct scores across raw score groups.
• Proposed a method to derive posterior predictive p-values for item fit statistics.

Main Results:

• Simulation studies and a real data application demonstrated the effectiveness of the proposed item fit diagnostics.
• The suggested techniques exhibited adequate statistical power for detecting item misfit.
• The Type I error rate of the proposed diagnostics was found to be reasonable.

Conclusions:

• The novel Bayesian item fit diagnostics, including the item fit plot and posterior predictive p-values, are effective for unidimensional IRT models with dichotomous items.
• These methods offer a promising advancement for psychometricians in evaluating item fit.
• The proposed techniques provide a statistically sound and practical approach to item fit assessment.