Improvement in Detection of Differential Item Functioning Using a Mixture Item Response Theory Model

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Differential item functioning (DIF) is typically detected by comparing item performance across observable groups.
• However, these manifest groups may not accurately represent the underlying source of bias.
• A latent variable approach is hypothesized to be more sensitive to true DIF.

Purpose of the Study:

• To evaluate the effectiveness of a latent DIF detection method compared to traditional manifest DIF detection.
• To investigate the performance of a mixture item response theory model with a latent grouping variable.
• To assess the impact of sample size, group proportions, and significance levels on DIF detection accuracy.

Main Methods:

• A simulation study was conducted to compare manifest and latent DIF detection methods.
• A mixture item response theory model incorporating a latent grouping variable was employed.
• The study analyzed varying sample sizes, relative group sizes, and significance levels.

Main Results:

• The latent DIF detection method, using a mixture item response theory model, demonstrated superior performance in identifying DIF items compared to manifest-only methods.
• The discrepancy between manifest and latent DIF detection widened as the correlation between the manifest variable and the true DIF source decreased.
• An empirical example using the General Aptitude Test Battery (GATB) Vocabulary test illustrated latent DIF detection in a minority sample.

Conclusions:

• Latent DIF detection offers enhanced sensitivity to bias compared to manifest DIF detection, particularly when manifest groups are imperfect proxies for the true source of DIF.
• The mixture item response theory model with a latent grouping variable is a valuable tool for more accurate DIF identification.
• This approach provides a more nuanced understanding of item bias and heterogeneity within test data.