Wald χ² Test for Differential Item Functioning Detection with Polytomous Items in Multilevel Data

Area of Science:

• Educational Measurement
• Psychometrics
• Statistics

Background:

• Equitable measurement in assessments requires identifying differential item functioning (DIF).
• Detecting DIF in multilevel data structures presents a significant challenge not fully addressed by existing methods.
• Polytomous items, common in many assessments, add complexity to DIF analysis.

Purpose of the Study:

• To introduce a novel procedure for detecting uniform and non-uniform DIF in polytomous items within multilevel data.
• To extend existing two-stage DIF detection procedures to accommodate multilevel data structures.

Main Methods:

• A Lord's Wald chi-squared test-based procedure was developed for DIF detection.
• The Metropolis-Hastings Robbins-Monro (MH-RM) algorithm was employed for estimating multilevel polytomous item response theory (IRT) models and covariance matrices.
• A two-stage approach was utilized, involving anchor item identification followed by candidate item evaluation.

Main Results:

• The proposed approach demonstrated high power in identifying DIF items.
• The method effectively controlled the Type I error rate in simulation studies.
• The procedure is robust under various simulated conditions mimicking real-world data.

Conclusions:

• The developed procedure offers a powerful and reliable method for detecting DIF in polytomous items within multilevel assessment data.
• This advancement contributes to more equitable measurement by providing tools to identify and address item bias in complex data structures.
• Further research should explore additional conditions and applications of this multilevel DIF detection method.