DIF Analysis with Unknown Groups and Anchor Items

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Ensuring fairness in survey questionnaires and educational tests is vital.
• Differential Item Functioning (DIF) analysis assesses item-level measurement invariance by detecting subgroup response differences.
• Traditional DIF methods require predefined reference/focal groups and anchor items, which are not always available.

Purpose of the Study:

• To propose a novel statistical framework for DIF analysis when both comparison groups and anchor items are unknown.
• To develop a method that simultaneously identifies latent subgroups and DIF items without prior information.
• To provide a robust approach for enhancing fairness in assessments.

Main Methods:

• A general statistical framework is proposed, modeling unknown groups via latent classes.
• Item-specific DIF parameters are introduced to capture differential item functioning.
• An L1-regularized estimator is employed to simultaneously identify latent classes and DIF items, assuming a small number of DIF items.
• A computationally efficient Expectation-Maximization (EM) algorithm is developed for the non-smooth optimization problem.

Main Results:

• The proposed L1-regularized method effectively identifies latent classes (unknown groups) and DIF items simultaneously.
• Simulation studies demonstrate the method's performance in various scenarios.
• The approach was successfully applied to real-world educational test data, validating its practical utility.

Conclusions:

• The developed framework offers a powerful solution for DIF analysis in the challenging setting where both group and anchor item information is missing.
• This method advances the assessment of measurement invariance and fairness in educational and psychological testing.
• The findings contribute to more equitable and reliable measurement instruments.