Identification of Differential Item Functioning in Multiple-Group Settings: A Multivariate Outlier Detection Approach

Area of Science:

• Psychometrics
• Educational Measurement
• Statistics

Background:

• Differential Item Functioning (DIF) is crucial for fair testing.
• Existing methods often struggle with more than two groups.
• Identifying DIF across multiple groups requires advanced statistical approaches.

Purpose of the Study:

• To develop a novel method for identifying DIF with multiple examinee groups.
• To conceptualize test items within a multivariate space for outlier detection.
• To provide a robust statistical technique for DIF analysis in complex group comparisons.

Main Methods:

• Treating test items as points in a multivariate space.
• Employing robust statistics to identify outlier items.
• Developing a graphical tool for low-dimensional (2-3 groups) analyses.
• Reanalyzing existing dataset on calculator use in mathematics tests.

Main Results:

• The proposed method effectively identifies DIF items as outliers.
• The multivariate space approach naturally accommodates multiple groups.
• Robust statistics enhance the reliability of DIF detection.
• The graphical tool offers a practical visualization for simpler cases.

Conclusions:

• The multivariate outlier detection method is a viable approach for multi-group DIF analysis.
• This technique offers a powerful alternative to traditional DIF detection methods.
• The study provides a framework for more nuanced fairness evaluations in educational assessments.