A Generalized Multi-Detector Combination Approach for Differential Item Functioning Detection.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Differential item functioning (DIF) detection is crucial for fair testing.
• Single detection methods (SDMs) for DIF have limitations due to unmet assumptions.
• Suboptimal SDM selection can reduce DIF detection accuracy.

Purpose of the Study:

• To introduce and validate a novel multi-detector combination (MDC) approach for DIF detection.
• To assess the accuracy and robustness of MDC compared to SDMs.
• To mitigate risks associated with selecting a single, potentially inappropriate, DIF detection method.

Main Methods:

• Developed an MDC approach integrating multiple SDMs using supervised learning.
• Applied five types of SDMs and four supervised learning methods within the MDC framework.
• Evaluated model performance using the area under the curve (AUC) metric.

Main Results:

• MDC methods consistently achieved higher average AUC values than SDMs.
• MDC demonstrated superior performance in both matched and unmatched test sets.
• MDC outperformed all individual SDMs across all tested conditions, indicating high accuracy and robustness.

Conclusions:

• The proposed MDC approach is a highly accurate and robust method for DIF detection.
• MDC effectively mitigates the limitations of relying on single detection methods.
• MDC offers a viable and improved solution for practical DIF analysis in various testing scenarios.