On the Complex Sources of Differential Item Functioning: A Comparison of Three Methods.

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Measurement

Background:

• Differential item functioning (DIF) is a persistent challenge in educational and psychological measurement.
• Complex DIF sources, where an item exhibits DIF across multiple variables simultaneously, complicate detection.
• Existing DIF detection methods, both non-Item Response Theory (IRT)-based and IRT-based, often fall short in evaluating complex DIF scenarios.

Purpose of the Study:

• To compare the performance of three DIF detection methods: logistic regression (LR), likelihood ratio test (LRT), and least absolute shrinkage and selection operator (LASSO) regularization.
• To evaluate these methods under conditions of complex DIF originating from multiple background variables.
• To provide insights into the factors influencing DIF detection accuracy in multi-variable contexts.

Main Methods:

• A comprehensive simulation study was conducted to compare LR, LRT, and LASSO.
• Empirical data analysis was performed to validate simulation findings.
• The study examined Type I error and Power rates of the methods under varying conditions.

Main Results:

• The performance of LR, LRT, and LASSO in detecting DIF on one variable was influenced by sample size, DIF magnitude, other variables' DIF magnitudes, and inter-variable correlations.
• LASSO regularization demonstrated promising results for detecting DIF across multiple background variables.
• Findings highlight the intricate interplay between multiple background variables and DIF detection.

Conclusions:

• The effectiveness of DIF detection methods is significantly impacted by the complexity of DIF sources and inter-variable relationships.
• LASSO regularization offers a viable approach for addressing complex DIF scenarios in psychometric and educational measurement.
• Further research is needed to explore limitations and refine DIF detection strategies for multi-variable contexts.