Detecting DIF with the Multi-Unidimensional Pairwise Preference Model: Lord's Chi-square and IPR-NCDIF Methods

Area of Science:

• Psychometrics
• Educational Measurement
• Noncognitive Assessment

Background:

• Multidimensional forced choice (MFC) measures are increasingly utilized in noncognitive assessment.
• Limited research exists on detecting differential item functioning (DIF) within these MFC models.

Purpose of the Study:

• To extend and evaluate two established DIF detection methods for MFC measures.
• To investigate the performance of Lord's chi-square and item parameter replication (IPR) methods within the Multi-Unidimensional Pairwise Preference (MUPP) model.

Main Methods:

• A Monte Carlo simulation was employed to examine Type I error rates and statistical power.
• Key variables manipulated included sample size, impact, DIF source (discrimination, threshold, location), and DIF magnitude.

Main Results:

• Both Lord's chi-square and IPR methods demonstrated consistent statistical power and controlled Type I error rates effectively across various conditions.
• Lord's chi-square showed superior performance when DIF originated from statement discrimination, while IPR was better for statement threshold DIF.
• Both methods performed comparably with better power when DIF originated from statement location.

Conclusions:

• Established DIF detection methods are suitable for use with the MUPP model in MFC tests.
• The choice between Lord's chi-square and IPR may depend on the specific source of DIF.
• Recommendations for practical application and limitations of DIF detection in MFC measures are provided.