A Log-Linear Modeling Approach for Differential Item Functioning Detection in Polytomously Scored Items.

Area of Science:

• Educational measurement
• Psychometrics
• Statistical modeling

Background:

• Differential item functioning (DIF) is crucial for test fairness.
• Log-linear models (LLM) are established for categorical data analysis.
• Evaluating DIF detection methods for polytomous items is essential.

Purpose of the Study:

• To assess the performance of the log-linear model (LLM) in detecting differential item functioning (DIF) for polytomous items.
• To compare LLM's DIF detection capabilities against other observed score-based methods.
• To investigate the impact of sample size, ability differences, and DIF types on detection accuracy.

Main Methods:

• Simulation study manipulating sample size, ability mean difference (impact), and DIF types.
• Comparison of log-linear model (LLM) with ordinal logistic regression, logistic discriminant function analysis, Mantel, and generalized Mantel-Haenszel.
• Evaluation based on Type I error (rejection rates) and power (DIF detection rates).
• LLM utilized 5 and 10 strata for observed score matching.

Main Results:

• Generalized Mantel-Haenszel and LLM with 10 strata demonstrated superior performance in DIF detection.
• Ordinal logistic regression and Mantel methods exhibited poor performance, particularly for balanced and partial DIF.
• LLM showed robust performance across various simulated conditions.

Conclusions:

• Log-linear models (LLM), especially with 10 strata, are effective for detecting differential item functioning (DIF) in polytomous items.
• Generalized Mantel-Haenszel is a strong alternative for DIF detection.
• Certain methods like ordinal logistic regression and Mantel are less suitable for complex DIF scenarios.