Score-Based Tests With Fixed Effects Person Parameters in Item Response Theory: Detecting Model Misspecification...

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Item response theory (IRT) models are widely used for educational assessment.
• Detecting model misspecification, such as differential item functioning (DIF), is crucial for valid test interpretation.
• Traditional methods for DIF detection can be computationally intensive and require complex model setups, especially for large-scale datasets.

Purpose of the Study:

• To introduce a fast, score-based statistical test for detecting model misspecification in IRT models.
• To provide a valid method for DIF screening when person parameters are treated as fixed effects, suitable for very large datasets.
• To develop a computationally efficient and simpler alternative to existing DIF screening methods.

Main Methods:

• Developed a score-based approximation test for IRT model misspecification.
• The test is compatible with constrained joint maximum likelihood (CJML) and joint maximum a posteriori (JMAP) estimators.
• Incorporated a spline-based residualization step to control Type I error rates when covariates correlate with ability.

Main Results:

• Simulations demonstrated nominal error rates and high power for detecting DIF with sufficient response data (15-20 responses per examinee).
• The method proved scalable and practical in an application to a large dataset (1,602 items, 57,684 students), identifying 13% of items with gender-related DIF.
• The proposed test showed high correlation with conventional DIF detection approaches.

Conclusions:

• The developed score-based test offers a robust, computationally light diagnostic for large-scale assessments.
• It is particularly valuable when classical random-effects approaches are infeasible or when ability group structures and DIF effect shapes are unknown or complex.
• This method enhances the efficiency and simplicity of DIF screening in psychometric analyses.