Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified

Area of Science:

• Educational Measurement
• Psychometrics
• Quantitative Psychology

Background:

• Cognitive diagnostic models (CDMs) are increasingly used in educational research for diagnostic insights.
• Investigating differential item functioning (DIF) is crucial for valid interpretations from CDMs.

Purpose of the Study:

• To evaluate the performance of three DIF detection methods within CDMs.
• To examine the specific impact of Q-matrix misspecification on these methods.

Main Methods:

• A simulation study was conducted to assess DIF detection methods.
• Methods evaluated included logistic regression, Mantel-Haenszel, and the Wald test.
• The influence of Q-matrix complexity and item structures was analyzed.

Main Results:

• Logistic regression and Mantel-Haenszel demonstrated superior Type I error control compared to the Wald test.
• Logistic regression and Wald methods exhibited higher statistical power.
• Method performance varied based on Q-matrix complexity and item characteristics.

Conclusions:

• A trade-off exists between Type I error control and statistical power in DIF detection.
• Q-matrix misspecification significantly affects the performance of DIF detection methods in CDMs.
• Careful consideration of Q-matrix properties is necessary when selecting DIF detection strategies.