Detecting uniform differential item functioning for continuous response computerized adaptive testing.

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing (CAT)

Background:

• Ensuring measurement fairness requires evaluating items for differential item functioning (DIF).
• Continuous response items offer more information than dichotomous items, particularly in performance-based tasks.
• Severe data sparsity is common in computerized adaptive testing (CAT) when items are machine-generated.

Purpose of the Study:

• To propose and evaluate two novel methods for detecting uniform DIF in the specific context of continuous response, severely sparse CAT.
• To assess the effectiveness of these methods in identifying DIF under challenging data conditions.

Main Methods:

• A modified non-parametric CAT-SIBTEST method, independent of item response theory (IRT) model assumptions.
• A parametric, model-based regularization method.
• Simulation studies were conducted to evaluate method performance.

Main Results:

• Both proposed methods demonstrated effectiveness in accurately identifying items exhibiting uniform DIF.
• The simulation studies confirmed the robustness of the developed techniques in the specified CAT scenario.

Conclusions:

• The developed CAT-SIBTEST modification and regularization method are suitable for detecting uniform DIF in continuous response, severely sparse CAT.
• These methods contribute to ensuring measurement fairness in advanced, data-intensive testing environments.
• A real data analysis is presented to illustrate practical application and potential limitations.