A Machine Learning Approach to Assess Differential Item Functioning in Psychometric Questionnaires Using the Elastic...

Area of Science:

• Psychometrics
• Statistical modeling
• Machine learning applications in social sciences

Background:

• Differential item functioning (DIF) assessment is crucial for test fairness.
• Ordinal logistic regression (OLR) is a common method for DIF detection.
• Maximum likelihood (ML) estimation in OLR can be biased with small sample sizes.

Purpose of the Study:

• To propose and evaluate an elastic net regularized OLR model for DIF assessment in small samples.
• To compare the performance of the regularized OLR model against the non-regularized OLR model.

Main Methods:

• A simulation study was conducted to compare DIF detection powers and Type I error rates.
• The study varied conditions such as DIF magnitude, sample size, sample size ratio, scale length, and weighting parameter.
• Elastic net regularization was applied to the OLR model.

Main Results:

• The elastic net regularized OLR model demonstrated increased power in detecting moderate and severe uniform DIF, particularly with small sample sizes (N=100, 150).
• For moderate DIF (0.4) and scale length (I=5), power increased by 35% and 21% for N=100 and N=150, respectively.
• For severe DIF (0.8) and scale length (I=10), power increased by approximately 29.3% and 11.2% for N=100 and N=150, respectively, with Type I errors near 0.05.

Conclusions:

• The elastic net regularized OLR model is a superior alternative to the non-regularized OLR model for DIF detection in small sample research.
• This study provides practical guidelines for researchers conducting DIF analyses with limited sample sizes.