The Matching Criterion Purification for Differential Item Functioning Analyses in a Large-Scale Assessment.

Area of Science:

• Psychometrics
• Educational Measurement

Background:

• Differential Item Functioning (DIF) is crucial for fairness in large-scale assessments.
• Accurate DIF detection ensures test items function similarly across different groups.

Purpose of the Study:

• To investigate the impact of matching criterion purification on DIF detection accuracy.
• To compare three matching approaches (block-level, pooled booklet, equated pooled booklet) with and without purification.

Main Methods:

• Employed the Mantel-Haenszel procedure with three matching approaches.
• Manipulated factors including test length, DIF item proportion, sample size, group ratio, and ability differences.
• Introduced systematic test form differences and considered purification strategies.

Main Results:

• Matching criterion purification generally enhanced the power of DIF detection across all three approaches.
• The extent of power improvement varied based on item characteristics and group ability differences.
• Purification slightly reduced Type I error rates when no mean ability difference existed.

Conclusions:

• Matching criterion purification is beneficial for improving DIF detection power.
• Pooled booklet matching and equated pooled booklet matching with purification are recommended for DIF analyses in large-scale assessments, balancing power and Type I error control.